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EXECUTIVE  SUMMARY 


Recent  advances  in  technology  and  rapid  changes  in  the  world 
have  placed  increasingly  stringent  demands  on  the  human  operator 
in  many  military  systems.  The  need  for  improved  and  more  varied 
skill  levels,  coupled  with  current  fiscal  constraints,  requires 
that  modern  military  training  systems  impart  the  complicated, 
higher-order  skills  required  to  operate  modern  combat  systems. 
Furthermore,  this  must  be  accomplished  in  less  time,  and  with  a 
lower  dollar  investment  than  in  recent  history.  Therefore,  the 
modern  training  challenge  demands  an  optimization  of  training 
resources- -a  return  on  investment  that  results  in  an 
uncompromisingly  high  level  of  readiness  at  the  lowest  possible 
cost,  and  in  the  shortest  time. 

OBJECTIVE 

The  purpose  of  the  present  research  was  to  advance 
understanding  of  effective  training  system  design  by 
investigating  factors  that  may  significantly  affect  the  success 
of  training  in  terms  of  performance  improvement  in  the 
operational  environment.  The  benefit  of  such  work  is  that  it  can 
lead  to  generalizable  training  design  guidelines  that  will 
increase  the  probability  of  effective  training,  with  a  relatively 
small  investment. 

A-iPROACH 

In  order  to  accomplish  this  goal,  a  comprehensive  model  of 
training  effectiveness  was  developed  by  synthesizing  several ^ 
diverse  literatures.  This  model  was  used  as  a  basis  to  specify 
testable  hypotheses.  A  large-scale  data  collection  effort  was 
then  conducted  with  Navy  recruits  as  an  initial  test  of 
predictions  from  the  model. 

RESULTS 

Results  indicated  that  several  "non- technical"  factors  had  a 
significant  impact  on  training  outcomes  in  this  setting.  These 
factors  included:  self-confidence,  task-related  attitudes, 
expectations  for  training,  training  fulfillment,  and  pre-training 
motivation. 

CONCLUSIONS 

Overall,  these  results  imply  that  no  matter  how  well 
designed  a  training  system  is,  training  effectiveness  will  not  be 
optimized  without  a  consideration  of  pertinent  individual  and 


5 


Technical  Report  93-011 


organizational  factors.  Therefore,  a  process  view  of  training 
effectiveness  should  yield  dividends  in  terms  of  an  improved 
understanding  of  crucial  training  variables,  and  in  turn, 
enhanced  training  outcomes.  The  framework  developed  here  can 
guide  future  research  and  continue  to  increase  our  understanding 
of  why  training  is  effective. 

RECOMMENDATIONS 

As  a  result  of  this  effort,  several  preliminary 
recommendations  for  training  can  be  offered.  These  include: 

1)  The  level  of  self-efficacy  of  trainees  should  be  assessed 
prior  to  training. 

2)  Remedial  training  to  raise  self-efficacy  levels  prior  to 
training  will  enhance  the  probability  of  positive  training 
outcomes . 

3)  Trainees  should  be  led  to  have  realistic  expectations  for 
training.  Interventions  to  meet  this  objective  should  be 
designed. 

4)  Interventions  designed  to  increase  trainee  commitment  to  the 
organization  will  enhance  the  likelihood  of  successful  training. 

5)  Efforts  to  improve  trainee  motivation  prior  to  training  can 
lead  to  better  training  outcomes. 
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INTRODUCTION 

Fleet  readiness,  safety,  and  performance  depend  largely  on 
the  extent  to  which  training  systems  impart  crucial  knowledge  and 
skills.  Further,  current  fiscal  constraints  demand  that  military 
training  resources  are  optimized- -that  is,  that  they  accomplish 
required  training  objectives  at  the  lowest  cost  and  in  the 
shortest  amount  of  time.  It  is  generally  agreed,  therefore,  that 
attention  must  be  directed  toward  understanding  the  factors  that 
influence  training  effectiveness  and  transfer  of  training,  so 
that  the  highest  payoff  in  terms  of  performance  improvement  can 
be  achieved. 

PROBLEM 

Past  research  into  training  system  design  has  most  often 
concentrated  on  a  relatively  small  set  of  variables,  such  as 
training  method,  content,  media,  and  equipment.  While  this 
research  is  important  (i.e.,  training  variables  are  a  crucial 
part  of  the  effectiveness  equation) ,  training  effectiveness  is  a 
complex  phenomenon.  There  are  numerous  factors  which  can 
influence  training  effectiveness  independent  of  training  quality. 
As  Goldstein  noted,  "we  must  consider  training  as  a  system  within 
work  organizations  rather  than  simply  treating  instruction  as  a 
separate  technology"  (1980,  p.  263) .  We  need  to  better 
understand  the  many  factors  that  may  contribute  to,  or  detract 
from,  training  effectiveness.  In  particular,  there  is  a  need  to 
examine  a  variety  of  often  overlooked  variables  in  the  training 
equation;  these  variables  include  trainee  attitudes, 
expectations,  and  motivations  (Noe,  1986) ,  and  organizational/ 
situational  factors  (e.g.,  supervisor  support  in  the  transfer 
environment)  (Noe  &  Schmitt,  1986) .  In  addition,  there  is  a  need 
to  apply  relevant  theories  to  guide  the  generation  of  hypotheses 
about  training  system  design,  and  to  provide  a  basis  upon  which 
to  make  design  decisions  (Cannon-Bowers,  Tannenbaum,  Salas,  & 
Converse,  1991)  . 

The  consequences  of  failing  to  specify  and  consider  all 
potentially  important  factors  in  training  system  design  (and 
perhaps  more  importantly,  the  relationship  among  factors)  are 
both  practical  and  theoretical.  From  a  theoretical  standpoint, 
neglecting  important  factors  in  the  training  equation  makes  it 
difficult  to  determine  why  training  may  or  may  not  have  been 
successful,  or  how  it  might  generalize  to  other  environments 
(Campbell,  1988;  Cannon-Bowers  et  al .  1991;  Tannenbaum  &  Yukl, 
1992) .  Related  to  this,  it  is  difficult  to  generate  general 
principles  of  training  system  design  since  it  is  unclear  why,  or 
by  what  mechanisms,  training  is  successful  or  unsuccessful.  On  a 
more  practical  level,  there  may  be  a  sub -optimization  of  training 
resource  allocation  and  expenditure,  and  of  training 
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effectiveness  since  design  decisions  are  not  based  on  sound 
principles  of  training.  For  example,  a  training  course  that 
fails  due  to  low  trainee  motivation  upon  entering  the  program, 
may  lead  a  designer  to  conclude  erroneously  that  the  failure  was 
due  to  the  training  methods  that  were  employed. 

Recently,  several  researchers  in  the  training  area  have 
contended  that  a  host  of  factors,  not  typically  considered  in 
training  design  research,  may  have  a  significant  impact  on 
training  effectiveness  (Noe,  1986;  Tannenbaum  &  Yukl,  1992)  .  In 
general,  these  factors  can  be  characterized  as  those  that  a 
trainee  brings  to  the  training  situation,  those  related  to  the 
training  system  itself,  and  those  stemming  from  the 
organizational  or  operational  context  in  which  the  training 
occurs.  Research  in  this  area  has  suggested  that  factors  such  as 
job  involvement,  performance  expectations,  training  fulfillment, 
career  planning,  and  organizational  favorability  can  all  have  an 
impact  on  training  effectiveness  (Mathieu,  Tannenbaum,  St  Salas, 
1992;  Noe  St  Schmitt,  1986;  Tannenbaum,  Mathieu,  Salas,  Sc 
Cannon-Bowers,  1991)  . 

Another  area  of  interest  to  the  current  research  relates  to 
the  need  to  define  the  concept  of  "training  effectiveness" 
itself.  Specifically,  it  has  been  typical  in  past  work  to  treat 
training  effectiveness  as  a  relatively  simple,  uni-dimensional 
construct.  A  notable  exception  here  is  the  theorizing  of 
Kirkpatrick  (1976) ,  where  the  concept  of  training  effectiveness 
was  decomposed  into  several  separate  outcomes:  reactions, 
learning,  behavior,  and  organizational  results.  According  to 
Kirkpatrick,  training  can  have  an  impact  on  any  (or  all)  of  these 
outcomes.  With  respect  to  the  current  research,  it  is  our 
contention  that  specifying  and  assessing  various  components  of 
training  effectiveness  is  crucial  to  a  full  understanding  of  how 
and  why  training  is  successful.  Moreover,  it  is  reasonable  to 
hypothesize  that  particular  training  system  features  will  have  a 
differential  impact  on  various  outcomes.  For  example,  trainees 
may  respond  favorably  to  a  training  program  without  actually 
learning  targeted  material,  or  they  may  learn  targeted  concepts 
but  be  unable  to  apply  these  to  the  job. 

The  purpose  of  the  current  research  was  to  extend  past  work 
in  the  training  effectiveness  area  by  specifying  a  comprehensive 
model  of  training  effectiveness,  and  studying  directly  the  impact 
of  selected  individual  and  situational  factors  on  various 
training  effectiveness  components  in  a  Navy  training  environment. 
Of  particular  interest  was  the  study  of  individual  factors, 
including  those  factors  that  a  trainee  brings  to  the  training 
program  which  affect  his/her  ability  to  acquire  and  apply 
targeted  skills,  and  how  these  affect  important  training 
outcomes . 
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OBJECTIVE 

The  objectives  of  the  current  research  were  to:  1) 
synthesize  several  diverse  literatures  in  order  to  develop  a 
comprehensive  model  of  training  effectiveness  that  would  provide 
a  framework  in  which  to  investigate  the  impact  of  training 
effectiveness  factors;  2)  determine  empirically  how,  and  to  what 
extent,  selected  training  effectiveness  factors  affect  training 
outcomes  in  a  military  training  environment;  and  3)  begin  to 
derive  recommendations  for  incorporating  knowledge  about  training 
effectiveness  factors  into  the  design  of  training  systems  as  a 
means  to  enhance  training  effectiveness. 

APPROACH 

A  series  of  research  questions  was  first  generated  to  guide 
subsequent  research  and  model  development.  These  included: 

1)  Which  organizational  and  individual  factors  are  likely  to 
affect  training  effectiveness? 

2)  What  are  the  important  components  or  categories  of  training 
effectiveness? 

3)  What  is  the  relationship  among  factors  that  affect  training 
effectiveness? 

4)  How  can  these  factors  be  measured  reliably? 

5)  What  is  the  impact  of  organizational  and  individual  factors  in 
an  actual  training  environment? 

6)  How  might  data  regarding  the  impact  of  these  factors  on 
training  effectiveness  be  used  to  improve  the  design  of  training 
systems? 

To  begin  to  answer  these  questions,  a  review  and  synthesis 
of  diverse  literatures  was  first  conducted.  This  included  the 
educational,  cognitive,  industrial,  and  social  psychology  litera¬ 
tures,  as  well  as  the  instructional  design  and  general  manage¬ 
ment/business  literatures.  Based  on  this  review,  a  comprehensive 
model  of  training  effectiveness  was  developed  in  order  to:  delin¬ 
eate  the  most  important  organizational  and  individual  factors 
that  are  hypothesized  to  affect  training  outcomes  (question  1 
above) ;  delineate  the  various  facets  of  training  effectiveness 
(question  2  above) ;  and  describe  how  these  variables  might  be 
related  to  one  another,  and  to  training  effectiveness  (question  3 
above) . 
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BACKGROUND:  A  MODEL  OF  TRAINING  EFFECTIVENESS 

A  detailed  review  of  the  diverse  literatures  that  are 
related  to  training  effectiveness  was  conducted.  As  a  result  of 
this  review,  a  model  of  training  effectiveness  was  developed, 
with  particular  attention  to  individual  characteristics, 
expectations,  and  motivation.  This  model  was  designed  to  reflect 
the  current  body  of  knowledge  regarding  training  effectiveness, 
and  to  guide  current  and  future  research  efforts.  It  is  not  a 
causal  model  per  se,  but  instead,  should  be  viewed  as  a  heuristic 
for  conceptualizing  and  examining  training  effectiveness. 

The  model  has  several  important  features.  First,  it  takes  a 
longitudinal,  process-oriented  perspective  that  considers  events 
that  occur  before,  during,  and  after  training,  and  their  effect 
on  training  effectiveness.  Second,  it  focuses  on  training  within 
the  organizational  or  work  context.  It  acknowledges  that  train¬ 
ing  does  not  occur  in  isolation  from  other  organizational  events. 
Third,  it  reconsiders  Kirkpatrick's  (1976)  training  evaluation 
typology,  yielding  a  revised  framework  with  greater  detail  and 
additional  training  outcomes  (more  will  be  said  about  this  in  a 
later  section) .  Fourth,  it  incorporates,  explicitly,  trainee 
expectations,  trainee  attitudes,  and  pre-  and  post-training 
motivation,  several  factors  that  have  tended  to  be  overlooked  in 
previous  research  on  training  effectiveness.  The  model  is  shown 
in  Figure  1 . 

Inspection  of  Figure  1  indicates  that  the  model  contains 
several  key  classes  of  variables  (e.g,  individual  variables, 
organizational/situational  characteristics) .  Some  of  these  are 
composed  of  several  variables  that  will  be  discussed  in  detail 
later  in  this  report.  Table  1  lists  all  of  the  relevant 
variables,  categorized  according  to  the  boxes  in  the  model. 

Moving  from  left  to  right,  the  model  hypothesizes  first  that 
individual  variables  (e.g.,  attitudes,  self-efficacy),  and 
organizational/situational  characteristics  (e.g.,  organizational 
climate,  trainee  selection  process)  influence  trainees'  expecta¬ 
tions  and  desires.  Similar  factors  influence  trainees'  motiva¬ 
tion  to  attend  training,  while  individual,  organizational,  and 
training  characteristics  influence  trainees'  motivation  to  learn. 

The  model  next  specifies  that  a  training  needs  analysis 
should  reflect  individual,  organizational,  and  task 
characteristics,  and  should  drive  the  training  method  and 
content.  Once  training  is  completed,  training  fulfillment 
becomes  crucial.  Training  fulfillment  is  defined  as  "the  extent 
to  which  the  training  met  trainees'  expectations  and  desires." 
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Figure  1:  Model  of  Training  Effectiveness 
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Table  1 

Variables  in  the  Trainina  Effectiveness  Model 


Individual  Characteristics  (Pre>Training) 

-  Abilities 

•  cognitive  ability 

•  psychomotor  ability 

•  learning  rates/trainability 

-  Attitudes 

•  commitment 

•  intent  to  remain 

•  career  planning 

•  job  satisfaction 

•  reactions  to  previous  training 

•  coworker/teammate  relations 

-  Self-Efficacy 

•  physical  self-efficacy 

•  cognitive  self-efficacy 

•  task-specific  self-efficacy 

-  Personality 

•  locus  of  control 

•  ego  strength 

•  need  for  achievement,  affiliation 

•  conformin' 

-  Demographics 

•  family  histoiy 

•  age 

•  gender 

•  education 

-  Experience 

•  tenure/experience  with  company 

•  with  task 

•  with  previous  training 

Organizational/Situational  Characteristics  (Pre-Training) 

-  Organizational  Climate 

•  participatory  versus  centralized 

-  Trainee  Selection/Notification  Process 

•  voluntary  vs.  mandatory  attendance 

•  reward  vs.  punishment 

•  communication  medium,  accuracy 

-  Purpose  of  Training 

•  maintenance  vs.  advancement 

-  Task  or  Job  Characteristics 

•  task  complexity 

•  task  type 

•  task  difficulty 

•  feedback 

-  Organizational  History' 

•  management-labor  relations 

•  growth/decline 

-  Organizational  Policies,  Programs,  &  Practices 

•  other  human  resource  practices 

•  other  company  practices 
Trainee  Expectations 

-  Training  Performance  Expectations 

-  Training  Expectations 

•  training  formal 

•  challenge 


•  degree  of  interactions 

•  focus  of  content 

-  Training  Desires 

•  training  format 

•  challenge 

•  focus  of  content 
Pre/During  Training  Motivation 

-  Motivation  to  Attend 

-  Motivation  to  Learn 
Training  Program  Characteristics 

-  Training  Needs  Analysis 

•  accuracy  of  need  identification 

•  involvement  of  potential  trainees 

-  Training  Method/Process 

-  Use  of  Training  Principles 

-  Training  Content 

-  Instructor  Characteristics 

-  Use  of  Technology 
Expectation  Fulfillment 

-  Perception/Expectation  Match 
Programmed  Interventions 

-  Relapse  Prevention 

-  Transfer  Support  Programs 
Training  Effectiveness 

-  Training  Reactions 

•  training  relevance/perceived  value 

•  affective  responses/happiness  index 

-  Learning 

-  Training  Performance 

-  Job  Performance 

-  Results/Organizational  Effectiveness 
Post-Training  Individual  Characteristics 

-  Attitudes 

•  commitment 

•  intent  to  remain 

•  job  satisfaction 

•  coworker/teammate  relations 

-  Ability 

•  task  specific  ability 

-  Self-Efficacy 

•  physical  self-efficacy 

•  cognitive  self-efficacy 

•  task  specific  self-efficacy 
Post-Training  Motivation 

-  Motivation  to  Transfer  and  Maintain 
Organizational/Situational  Variables  -  Post  Training 

-  Transfer  Environment 

•  supervisor  support 

•  co-worker  support 

•  resource  availability  (time,  equipment) 

•  workload 

•  job  security 

•  authority/autonomy 

-  Organizational  Culture 

•  openness  to  innovation/risk  taking 
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It  is  directly  related  to  training  reactions;  more  specifically, 
training  that  meets  or  exceeds  expectations  and  desires  should 
exhibit  more  positive  trainee  reactions. 

Learning  is  a  function  of  the  following:  training  content, 
method,  and  process;  trainees'  motivation  to  learn;  and  trainee 
ability.  Training  and  ability  may  interact  in  determining 
learning.  Training  performance  is  a  function  of  training  content 
and  method,  learning,  and  trainee  ability. 

Post -training  motivation  is  influenced  by  post- training 
organizational /situational  characteristics  (e.g.,  supervisor  and 
peer  support)  and  any  maintenance  interventions  (e.g.,  relapse 
prevention  programs) .  Job  performance,  or  transfer,  is  a 
function  of  training  performance  moderated  by  post -training 
motivation  to  transfer,  as  well  as  post-training  organizational/ 
situational  characteristics  (e.g.,  resource  availability).  The 
rationale  is  that  trainees  who  can  perform  the  task  during 
training  will  also  perform  it  back  on  the  job  if:  (1)  they  want 
to,  and  (2)  they  have  the  necessary  resources. 

Finally,  results/organizational  effectiveness  is  a  function 
of  job  performance,  moderated  by  the  accuracy  of  the  training 
needs  analysis.  The  rationale  here  is  that  behavior  change 
resulting  from  training  should  contribute  to  organizational 
effectiveness  to  the  extent  that  the  training  addressed  the 
appropriate  organizational,  individual,  and  task  needs. 

Jr  should  be  noted  that  the  attitudes,  skills,  learning,  and 
organizational  changes  that  result  from  a  given  training  progpm 
will  serve  as  antecedents  of  expectations,  desires,  and  training 
motivation  in  subsequent  training  programs. 

MEASURING  TRAINING  EFFECTIVENESS 

As  noted,  the  construct  of  training  effectiveness  has  often 
been  treated  as  a  simple,  uni -dimensional  construct  in  past  work. 
However,  it  is  our  contention  that  training  effectiveness  is  a 
more  complex  construct,  with  several  facets  or  components. ^  The 
following  sections  expand  upon  these  notions  regarding  training 
effectiveness . 

To  begin  with,  training  effectiveness  can  be  defined  as  the 
extent  to  which  training  yields  desired  or  relevant  outcomes. 
Training  effectiveness  is  usually  assessed  via  a  training 
evaluation  study,  which  involves  comparing  post-training 
pejfformance  to  a  specified  criterion  or  standard.  There  is  not  a 
single,  all-encompassing,  universally  accepted  training 
effectiveness  criterion,  nor  should  there  be.  Different  training 
programs  have  different  goals  and  processes,  and  thus  require 
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different  measures  of  training  effectiveness.  However,  while  the 
specific  measures  may  vary,  it  is  possible  to  categorize 
effectiveness  measures  on  the  basis  of  similar  features. 

D.L.  Kirkpatrick  (1959a,  1959b,  1960a,  1960b,  1976)  proposed 
a  typology  of  training  evaluation  that  partitioned  training 
effectiveness  into  four  steps.  The  four  steps  are: 

1)  Reactions :  How  well  did  the  trainees  like  the  program?  What 
were  their  feelings  about  the  training? 

2)  Learning :  What  principles,  facts,  and  techniques  were 
learned,  understood,  and  absorbed  by  trainees? 

3)  Behavior:  What  changes  in  job  behavior  resulted  from  the 
training?  Were  trainees  using  learned  principles  and  techniques 
on  the  job? 

4)  Results :  What  were  the  tangible  results  of  the  program  in 
terms  of  reduced  cost,  improved  quality,  improved  quantity,  and 
so  forth? 

Kirkpatrick's  typology  has  helped  guide  numerous  research 
and  training  evaluation  efforts,  and  is  probably  the  most  fre¬ 
quently  cited  framework  for  understanding  training  effectiveness. 
The  usefulness  and  power  of  Kirkpatrick's  model  has  been  its 
simplicity,  and  its  ability  to  help  people  think  about  training 
criteria  (Alliger  &  Janak,  1989) .  However,  in  some  respects,  it 
lacks  sufficient  detail  or  is  ambiguous,  and  it  fails’  to  consider 
other  possible  training  outcomes. 

We  are  proposing  a  revision  to  Kirkpatrick's  typology  that 
addresses  more  fully  the  training  effectiveness  criterion  space. 
It  has  particular  relevance  to  the  way  the  military  evaluates  its 
training,  but  should  also  be  generalizable  to  other  training 
environments.  The  six  proposed  categories  of  training 
effectiveness  are: 

1)  Reactions 

2)  Attitude  Change 

3)  Learning 

4)  Training  Performance  (Behavior  I) 

5)  Job  Performance  (Behavior  II) 

6)  Results/Organizational  Effectiveness 
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All  six  categories  are  depicted  in  Figure  1.  With  the 
exception  of  attitude  change,  each  is  shown  explicitly.  Attitude 
Change  is  implied  in  two  locations:  1)  it  can  be  considered  part 
of  the  post -training  "individual  characteristics"  box,  and  2)  it 
is  part  of  the  "post-training  motivation"  box.  Each  category  of 
training  effectiveness  is  addressed  in  detail  below. 


Reactions 

This  category  is  similar  to  Kirkpatrick's  Reactions 
category.  However,  "Reactions"  is  probably  best  thought  of  as  a 
multidimensional  construct.  Specifically,  it  includes  an 
affective  response,  or  liking,  component  (including  an  assessment 
of  hygiene -type  factors,  such  as  length  of  training  and 
conditions)  as  well  as  trainee  perceptions  of  the 
usefulnes.q /relevance  and  perceived  value  of  the  training.  It 
should  be  noted  that  Reaction  measures  are  the  most  common  form 
of  training  evaluation  (Brown,  1980;  Saari,  Johnson,  McLaughlin, 

&  Zimmerle,  1988;  Swierczek  &  Carmichael,  1985). 

Attitude  Change 

Trainees  may  leave  training  with  different  perspectives  than 
when  they  entered.  The  training  experience  may  have  an  effect  on 
trainees'  self-efficacy  (Gist,  1987),  attitudes  toward  teamwork 
or  quality  (Helmreich,  Foushee,  Benson  &  Russini,  1986), 
motivation  (Latham,  1989),  commitment  (Tannenbaum,  et  al . ,  1991), 
and  intent  to  remain  with  the  organization,  to  name  just  a  few 
possible  effects.  These  changes  are  referred  to  broadly  as 
Attitude  Chancre.  In  face,  some  training  programs  are  designed 
with  attitude  change  as  a  primary  focus.  For  example,  several 
airlines  routinely  conduct  crew  coordination  training  where  the 
primary  focus  is  on  changing  crew  members  attitudes  towards 
teamwork  (see  Prince  &  Salas,  1993) .  Moreover,  some  management 
training  is  designed  specifically  to  foster  a  change  in 
organizational  climate  (Moxnes  &  Eilertsen,  1991) . 

Some  training  is  designed  to  affect  motivation  or  trainee 
resource /effort  allocation,  rather  than  skill  acquisition,  al¬ 
though  it  is  likely  that  training  that  affects  both  effort  and 
skill  would  have  the  greatest  impact.  Nonetheless,  trainee 
motivation  after  completing  training  could  be  an  appropriate 
index  of  training  effectiveness  in  some  instances.  For  example, 
according  to  a  meta-analytic  review,  for  the  most  part,  training 
designed  to  change  motivation  and  values  in  supervisors  does 
appear  to  do  so  (Burke  &  Day,  1986) . 

Another  important  attitude  that  can  be  affected  by  training 
is  self-efficacy.  Self-efficacy  refers  to  the  belief  in  one's 
capability  to  perform  a  specific  task  (Bandura,  1977) ,  and  has 
been  shown  to  be  related  to  subsequent  performance  (Barling  & 
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Beattie,  1983;  Taylor,  Locke,  Lee,  &  Gist,  1984).  In  fact, 
behavior  modeling  may  be  an  effective  training  method  because  of 
its  impact  on  self-efficacy.  Further,  changes  in  self-efficacy 
are  considered  a  key  part  of  the  cognitive-behavioral  relapse 
process  (Marx,  1982) . 

Along  this  line,  a  study  by  Hill,  Smith,  and  Mann  (1987) 
confirmed  the  Bandura  (1977)  findings  that  performing  a  behavior 
otherwise  thought  to  be  impossible  is  likely  to  increase  self- 
efficacy,  and  further  revealed  that  experience  is  not  likely  to 
influence  decisions  to  learn  about,  or  use  newly  learned  skills, 
unless  self-efficacy  has  been  affected.  In  other  words, 
post- training  self-efficacy  should  affect  trainees'  motivation  to 
transfer  and  to  use  newly  acquired  skills  and  knowledge.  For 
this  reason,  several  authors  have  noted  that  post -training 
self-efficacy  should  be  considered  an  important  outcome  of 
training,  and  one  potential  indication  of  training  effectiveness 
(Gist,  1987;  Latham,  1989;  Tannenbaum  &  Yukl,  1992) .  That  is,  to 
the  extent  that  training  results  in  increased  trainee 
self-efficacy,  it  may  be  deemed  effective. 

Furthermore,  to  the  extent  that  self-efficacy  generalizes 
across  situations,  referred  to  as  "generality"  by  Bandura  (1977) , 
it  could  yield  additional  dividends  by  improving  subsequent 
performance  on  non-trained  tasks  as  well.  Kanfer  and  Ackerman 
(1989)  concluded  that  "additional  research  to  clarify  the 
determinants  of  transfer  of  self  confidence  expectations  across 
tasks  has  important  practical  implications  for  training"  (p. 

686)  . 


Training  can  also  have  an  impact  on  attitudes  such  as  satis¬ 
faction  and  organizational  commitment.  In  fact,  an  investment 
model  based  on  exchange  theory  (see  Farrell  &  Rusbult,  1981) 
would  suggest  that  training  could  be  considered  an  organizational 
investment  in  its  employees,  and  actually  viewed  as  a  reward  by 
some  employees.  To  the  extent  that  training  at  company  X  is 
viewed  as  valuable  and  better  than  company  Y,  theoretically,  it 
could  add  to  satisfaction  and  commitment .  Empirically,  Louis, 
Posner,  and  Powell  (1983)  found  a  positive  relationship  between 
'  perceptions  that  training  was  helpful,  and  employee  satisfaction 
and  commitment . 

Learning 

As  conceptualized  here,  learning  is  a  cognitive  process 
referring  to  the  acquisition  of  knowledge.  Learning  may  be 
manifested  in  the  amount  of  knowledge  acquired,  or  in  the 
structure  of  the  knowledge  acquired  (see  Goldsmith,  Johnson,  & 
Acton,  1991;  Kraiger,  Ford,  &  Salas,  1993) .  Learning  does  not 
imply  that  the  trainee  can  perform  a  task  differently,  but  simply 
that  he/she  has  acquired  knowledge  with  which  to  perform  a  task 
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differently.  It  addresses  questions  such  as:  can  trainees  recite 
new  information  after  training?;  and  can  they  verbalize  new 
strategies,  concepts,  or  approaches  to  performing  a  task? 

The  cognitive  psychology  and  learning  literatures  have 
delineated  different  aspects  of  the  learning  process,  including 
the  acquisition  of  declarative  knowledge  (the  "what"  component) , 
procedural  knowledge  (the  "how"  component) ,  and  conditional 
knowledge  (the  "when  and  why"  component)  (Anderson,  1985; 
Cassidy-Schmitt  &  Newby,  1986;  Kanfer  &  Ackerman,  1989).  These 
may  be  assessed  at  the  Learning  level  of  training  effectiveness 
by  constructing  knowledge  tests,  or  they  may  be  assessed  as  part 
of  behavior  change.  Kyllonen  and  Shute  (1989)  presented  a 
taxonomy  of  learning  skills  that  may  be  of  value  in  considering 
the  types  of  learning  that  can  be  measured.  Quite  recently, 
Kraiger  et  al.  (1993)  expanded  the  concept  of  learning  measures 
to  incorporate  several  more  cognitively-oriented  assessment 
devices. 

Burke  and  Day  (1986) ,  in  their  meta-analysis  of  management 
training  effectiveness,  partitioned  learning  into  subjective 
learning,  i.e.,  principles,  facts,  attitudes,  and  skills  that 
were  learned  as  communicated  in  statements  _  of  opinicpn,  belief,  or 
judgment  by  trainee  or  trainer,  and  objective  learning,  i.e. , 
knowledge  assessed  through  knowledge  tests  and  other  related 
measures.  Subjective  learning,  as  they  define  it,  would  fall 
into  our  category  of  training  Reactions  -  releVance/perceived 
value.  In  general,  our  learning  category  is  consistent  with 
Burke  and  Day's  (1986)  objective  learning  category. 

Training  Performance 

In  an  expansion  of  Kirkpatrick's  typology  we  partition 
behavior  into  two  categories:  (1)  Training  Performance,  and  (2) 
Transfer  Behavior.  In  contrast  to  the  Learning  category,  both 
denote  that  the  trainee  can  perform  the  task  differently,  thereby 
incorporating  the  demonstration  or  execution  of  behavior  change. 
Training  Performance  assesses  behavior  change  prior , to  the 
transfer  environment .  Transfer  Behavior  assesses  behavior  change 
after  returning  to  the  job. 

Training  Performance  goes  beyond  Learning  by  requiring  that 
trainees  show  that  they  can  incorporate  the  knowledge  they  have 
acquired  into  their  actions.  The  requisite  skills  and  abilities 
needed  to  demonstrate  Training  Performance  and  Learning  may  be 
different.  For  example,  a  medical  trainee  may  be  able  to  recall 
the  steps  for  a  particular  surgical  procedure  (Learning) ,  but  may 
lack  the  manual  dexterity  to  perform  the  procedure  during  a 
simulated  operation  (Training  Performance) .  Training  Performance 
can  be  measured  through  the  use  of  role  plays,  simulations,  or 
work  samples.  For  tasks  that  have  only  a  cognitive  element  and 
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do  not  have  a  behavioral  component,  the  distinction  between 
Learning  and  Training  Performance  may  be  irrelevant. 

To  further  clarify,  Training  Performance,  as  defined  here, 
approximates  more  of  a  "maximum"  performance  criterion  than  a 
"typical"  performance  criterion.  Maximum  performance  measures 
are  characterized  by:  (1)  an  explicit  awareness  of  being 
evaluated,  (2)  an  acceptance  of  explicit  instructions  to  maximize 
effort,  and  (3)  a  short  enough  measurement  period  to  allow 
focused  attention  on  the  goal  (Sackett,  Zedeck,  &  Fogli,  1988)  . 
They  reflect  the  fact  that  the  trainee  is  doing  his/her  "best" 
for  the  purpose  of  demonstrating  mastery  of  targeted  material. 

In  contrast,  typical  performance  criteria  refer  to  the  level  of 
performance  that  would  be  displayed  by  a  trainee  when  he/ she  is 
not  being  evaluated  explicitly  (i.e.,  during  routine  or  typical 
performance  sessions) .  Maximum  and  typical  performance  criteria 
reflect  somewhat  different  phenomena  (Sackett,  et  al . ,  1988). 

For  most  military  situations.  Training  Performance  is  the _ 
highest  level  of  training  effectiveness  measure  assessable  during 
peace  time.  The  Government  Accounting  Office  (GAO)  (June,  1986) 
noted  that,  "Most  military  officials  we  interviewed  consider 
joint  exercises,  such  as  the  annual  'Return  of  Forces  to  Germany 
and  combined  arms  and  interservice  training. . .  (and  training 
centers  and  ranges) . . .  to  be  the  best  evaluations  of  unit 
performance"  (p.  14) .  The  Navy  conducts  Operational  Training 
Assessments  to  determine  how  well  training  has  prepared  the  ship 
and  its  crew  for  deployment.  These  simulations  can  be  viewed  as 
the  highest  level  of  Training  Performance  (Behavio;.*  I )  measures 
possible,  and  may  be  particularly  useful  for  assessing  team 
training  effectiveness.  However,  these  exercises  only  simulate 
combat.  True  combat  situations  add  considerable  stress,  and  are 
inherently  inappropriate  for  collecting  training  effectiveness 
data . 

Transfer  Behavior 

As  with  Training  Performance,  Transfer  Behavior  implies 
Lehavior  change.  However,  it  goes  beyond  Training  Performance. 
Training  Performance  assesses  the  question,  can  the  trainee 
p03fform  the  task  differently?  Transfer  Behavior  assesses  the 
q\i0stion,  does  the  trainee  perform  the  task  differently  after 
he/she  has  returned  to  the  job?  The  former  reveals  behavioral 
capability,  the  later  behavioral  change. 

As  Baldwin  and  Ford  (1988)  noted,  transfer  of  training  to 
the  job  includes  the  generalization  of  learned  material  to  the 
job,  as  well  as  the  maintenance  of  training  skills  over  a  period 
of  time  on  the  job.  The  generalization  of  learned  material 
constitutes  two  forms  of  transfer:  (1)  vertical  and  (2)  lateral 
(Gagne  &  Smith,  1967)  .  The  integration  of  subskills  into  higher 
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level  skills  is  vertical  transfer.  Thus,  the  trainee  may  learn 
and  demonstrate  several  "subskills"  during  training.  The  ability 
to  pull  those  together  into  a  higher  level  skill  and  apply  it  to 
the  job,  is  an  example  of  vertical  transfer.  Applying  the  newly 
developed  skills  in  the  appropriate  situations  is  lateral  trans¬ 
fer  .  The  cognitive  skills  necessary  for  vertical  and  lateral 
transfer  may  be  incorporated  into  training,  or  may  be  conveyed 
and  developed  after  training.  Other  aspects  of  transfer 
generalization  include  transfer  distance  (Laker,  1990) ,  literal 
and  figural  transfer,  and  specific  and  non-specific  transfer 
(Ford,  1990) . 

The  job  environment  is  always  somewhat  different  from  the 
training  environment.  A  trainee  may  be  able  to  focus  on  one 
primary  task  during  training,  but  usually  must  balance  multiple 
tasks  as  part  of  his  job.  Furthermore,  upon  returning  to  work, 
trainees  may  find  they  do  not  have  the  necessary  time,  resources, 
support,  or  opportunity  to  practice  new  skills  (Ford,  Quinones, 
Sego,  &  Sorra,  1992)  .  Transfer  Behavior  reflects  behavioral 
change  given  the  various  constraints  or  facilitators  that  may 
exist  in  the  job  environment.  It  requires  not  only  that  trainees 
have  acquired  the  capability  to  perform  the  task  differently,  but 
also  that  they  are  motivated  to  apply  their  learning  to  the  job, 
and  have  the  resources  to  do  so.  In  fact,  several  researchers 
have  noted  that  when  trainees  lack  conditional  knowledge  (i.e., 
knowing  why  they  are  learning  something  or  the.  significance  of 
the  skill) ,  their  effort  to  maintain  and  generalize  the  skill 
quickly  diminishes  (Brown  &  Palinscar,  1982;  Belmont  &  Butter¬ 
field,  1971;  Kendall,  Borkowski,  &  Cavanaugh,  1980).  Conditional 
knowledge  may  be  conferred  during  training,  or  may  come  later 
from  the  trainee's  supervisor. 

A  training  program  may  lead  to  trainee  Learning  and  Training 
Performance,  but  due  to  constraints  in  the  transfer  environment, 
may  fail  to  demonstrate  Transfer  Behavior  changes.  One  could 
argue  that,  in  that  instance,  the  training  was  effective  and  was 
not  the  problem.  However,  when  training  is  viewed  in  an 
organizational  context  (and  not  in  isolation) ,  we  must  conclude 
that  training  that  does  not  transfer  was  not  completely 
effective,  and  that  interventions  should  be  targeted  to  facili¬ 
tate  the  transfer  process. 

Results /Organizational  Effectiveness 

This  category  is  similar  to  Kirkpatrick's  Results  step. 
Results  refer  to  quantifiable  changes  in  related  outcomes  as  a 
result  of  trainees'  behavioral  changes.  For  example,  a  trainee 
could  return  to  his/her  job  and  perform  a  particular  machining 
task  differently  (Transfer  Behavior) ,  resulting  in  reduced  waste 
(Results) .  However,  it  is  possible  that  behavioral  changes  may 
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not  yield  changes  in  results,  or  may  yield  undesirable  changes  in 
results . 

According  to  Kirkpatrick  (1976) ,  other  examples  of  results 
are  reduced  grievances,  increased  quantity,  reduced  turnover 
(also  noted  by  Horrigan,  1979) ,  and  reduced  costs.  Safety  may  be 
either  a  behavior  or  a  result,  depending  upon  how  it  is  measured. 
Reber  and  Wallin  (1984)  used  safety  as  a  measure  of  behavior 
change  by  observing  and  recording  the  incidence  of  specific 
safety  behaviors  (e.g.,  wearing  safety  glasses).  Alternatively, 
an  examination  of  increases  or  decreases  in  the  number  of 
accidents  would  be  a  safety  measure  that  corresponds  to  the 
Results  criterion  of  training  effectiveness. 

What  is  implied  by  Kirkpatrick's  Results  category  is  that 
the  appropriate  results  have  been  identified,  and  that  the 
results  are  in  fact  related  to  Organizational  Effectiveness.  We 
want  to  make  this  assumption  more  explicit ,' since  it  has 
implications  for  the  conclusions  that  are  drawn  regarding 
training  effectiveness.  If  training  is  designed  to  be  consistent 
with,  and  support  the  attainment  of,  organizational  results,  and 
these  results  are  actually  important  to  organizational 
effectiveness,  then  improvement  in  organization-level  variables 
(as  a  function  of  training)  can  be  expected. 

On  the  other  hand,  if  training  is  not  properly  linked  to 
organizational  objectives,  or  if  desired  results  do  not 
necessarily  lead  to  improved  organizational  effectiveness,  then 
training  may  have  no  impact  on  the  "bottom  line,"  or  may  actually 
reduce  effectiveness.  For  example,  if  training  has  unduly 
shifted  employees'  attention  away  from  critical  aspects  of  their 
job  toward  less  important  aspects,  we  might  see  that  changes  in 
Behavior  could  lead  to  inappropriate  changes  in  Results. 

Consider,  for  instance,  a  training  course  designed  to  enhance 
cleanliness  aboard  ship.  Due  to  the  training,  trainees 
demonstrate  changes  in  cleaning  behaviors  (Transfer  Behavior)  and 
cleanliness  aboard  ship  improves  (Results) .  However,  the 
trainees  now  spend  a  disproportionate  amount  of  their  time 
focusing  on  cleaning  behaviors,  neglecting  more  critical  aspects 
of  their  job;  Organizational  Effectiveness  declines. 

This  is  another  example  of  how  examining  training  in 
isolation  can  be  misleading.  In  isolation,  this  training  appears 
quite  successful.  But,  examined  in  the  larger  organizational 
context,  this  training  has  deleterious  effects.  It  is  the  entire 
training  system  (e.g.,  the  mix  of  courses  taken),  as  well  as 
other  human  resource  and  company  policies,  programs,  and 
experiences,  that  provide  an  individual  with  information  about 
the  appropriate  weightings  of  job  tasks. 
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The  selection  and  measurement  of  relevant  Results  criteria 
should  flow  from  a  systematic  training  needs  analysis,  including 
an  organizational  analysis  that  explicitly  considers  organiza¬ 
tional  goals  (Goldstein,  1993;  Wexley  &  Latham,  1981).  Training 
needs  analysis  can  strengthen  the  link  between  Transfer  Behavior 
and  Results/Organizational  Effectiveness  by  ensuring  that  the 
appropriate  behaviors  have  been  targeted  for  change.  Bownas, 
Bosshardt,  and  Donnelly's  (1985)  and  Ford  and  Wroten's  (1984) 
research  on  content  evaluation  of  training  are  good  steps  towards 
assessing  and  ensuring  this  match. 

Assessing  Results  can  be  quite  difficult  for  many  types  of 
training.  As  Kirkpatrick  (1976)  noted,  "there  are  however  so 
many  complicating  factors  that  it  is  extremely  difficult,  if  not 
impossible,  to  evaluate  certain  kinds  of  programs  in  terms  of 
their  results.  Therefore,  it  is  recommended  that  training  direc¬ 
tors  evaluate  in  terms  of  reactions,  learning,  and  behaviors 
(p.  21) .  In  an  interesting  study,  Russell,  Terborg,  &  Powers 
(1984)  measured  the  relationship  between  use  of  sales  training 
(as  measured  by  the  percent  of  store  personnel  that  received 
training  and  the  perceptions  of  training  emphasis)  and 
0valuations  of  store  performance.  Their  study  is  a  rare  example 
of  using  an  organizational  level  performance  measure  in  an 
attempt  to  assess  training  effectiveness. 

A  GAO  study  (June,  1986)  addressed  organizational -level 
measures  of  training  performance  for  the  military.  They  noted 
that  the  Department  of  Defense  defines  readiness  as  "the  ability 
of  forces,  units,  weapons  systems  or  equipment  to  deliver  the 
outputs  for  which  they  were  designed  (including  the  ability  to 
(;j0ploy  and  employ  without  unacceptable  delays)  .  "  Clearly, 
readiness  could  be  a  high  level  criterion  for  training 
effectiveness.  Yet,  the  GAO  reported,  "Although  a  units' 
readiness  is  heavily  influenced  by  the  amount,  type,  and  quality 
of  training  it  receives,  the  services  cannot  determine  precisely 
how  readiness  is  affected  by  changes  in  the  level  of  training 
n  (p •  2)  •  It  is  oftsn  difficult  to  3.ss0ss  t^rsiriiri^^ 
ef  f  6ct iv6n0ss  in  tomns  of  Rosults/Organiza-tiona.!  Ef  f 0Ctiv0n0ss  . 

Relationship  Among  Training  Outcomes 

As  Alliger  and  Janak  (1989)  noted,  previous  researchers  and 
practitioners  have  made  certain  assumptions  about  the  relation¬ 
ship  among  Kirkpatrick's  levels  of  training  effectiveness  (see 
Hamblin,  1974) .  They  have  assumed  that  the  levels  are  causally 
linked  (e.g.,  reactions  are  causally  linked  to  learning) ,  and 
that  the  relationship  among  them  is  positive  (i.e.,  positive 
reactions  lead  to  better  learning) .  In  fact,  Kirkpatrick  s 
typology  is  sometimes  referred  to  as  a  hieT3.Tchy ^  of  tz'sining 
effectiveness  to  reflect  this  assumed  relationship. 
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As  depicted  in  the  framework  (see  Figure  1) ,  it  is 
hypothesized  that  the  relationship  among  effectiveness  components 
may  not  be  this  straight  forward.  Specifically,  we  believe  that 
there  is  a  direct  link  among  several,  but  not  all,  of  the 
training  effectiveness  measures,  and  suggest  further  that  a 
number  of  variables  may  moderate  these  relationships.  In 
particular,  we  hypothesize  a  hierarchical  link  from  learning  to 
training  behavior,  from  training  behavior  to  transfer  behavior, 
and  from  transfer  behavior  to  results/organizational 
effectiveness.  This  conceptual  hierarchy  is  based  on  the 
following  logic:  learning  is  a  prerequisite  to  training 
performance  to  the  extent  that  training  performance  requires  the 
use  of  knowledge  acquired  during  training. 

However,  training  performance  has  another  prerequisite; 
trainees  must  possess  the  skills  and  abilities  necessary  to 
perform  the  trained  behaviors.  Thus,  a  trainee  may  be  able  to 
recite  the  appropriate  strategies  for  dealing  with  an  approaching 
aircraft,  but  when  confronted  in  a  simulation,  may  lack  the 
composure  or  verbal  skills  needed  to  behave  appropriately. 

Without  learning  what  to  do  differently,  changes  in  behavior  are 
impossible.  However,  simply  knowing  what  to  do  does  not  imply 
that  the  trainee  can  do  it.  In  other  words,  learning  is  a 
necessary,  but  not  sufficient,  condition  for  behavior  change  to 
occur.  Similar  arguments  could  be  made  for  the  relationship 
between  training  performance  and  transfer  behavior,  and  between 
transfer  behavior  and  results.  That  is,  if  trainees  cannot 
perform  the  trained  skill  under  training  conditions  (i.e., 
maximum  performance) ,  it  is  unlikely  that  they  will  be  able  to 
perform  them  as  part  of  their  regular  job  duties.  However, 
simply  because  they  can  demonstrate  the  behavior  during  training 
does  not  mean  that  they  will  use  them  on  the  job.  Likewise,  if 
trainees  do  not  behave  differently  on  the  job,  organizational 
results  cannot  improve.  Additionally,  all  changes  in  behavior  do 
not  have  a  positive  effect  on  results. 

Previous  research  has  not  always  confirmed  the  hierarchical 
relationships  suggested  here  (See  Alliger  and  Janak,  1989)  . 

There  are  three  possible  explanations  for  this  failure.  First, 
the  proposed  hierarchy  may  be  invalid.  However,  some  empirical 
support  has  been  reported  (e.g.,  Latham,  Wexley,  &  Purcell, 

1975) ,  and  the  logic  behind  the  hierarchy  appears  sound.  Second, 
as  we  suggested,  trainee  accomplishment  of  a  previous  level  may 
be  a  necessary,  but  not  sufficient,  condition  for  accomplishment 
on  the  next  level.  Different  variables  contribute  to  the 
attainment  of  each  set  of  outcomes.  Specifically,  in  some  cases 
they  act  as  moderators  in  the  relationship  among  the  criteria, 
and  reduce  the  correlations  among  outcomes  (e.g.,  supervisor 
support  moderating  the  relationship  between  training  performance 
and  transfer  behavior) .  Third,  the  failure  of  some  studies  to 


27 


Technical  Report  93-011 


support  the  hierarchy  may  be  the  result  of  the  way  in  which 
training  outcomes  have  typically  been  measured. 

Measurement  issues  can  complicate  the  interpretation  of 
observed  correlations  between  outcomes.  It  is  not  uncommon  for 
organizations  to  employ  learning  measures,  training  performance 
measures,  and  transfer  behavior  measures  that  measure  different 
training  objectives,  or  that  are  not  related  to  the  training 
objectives  at  all.  For  example,  in  tactical  decision  making 
training,  the  learning  measure  could  focus  on  appropriate 
responses  to  a  particular  type  of  air  contact,  the  training 
performance  measure  could  be  a  simulation  that  incorporates  a 
wide  range  of  behaviors  (including  responding  to  air  contact,  but 
not  limited  to  it) ,  and  the  transfer  behavior  measure  could  be  a 
global,  pre-post  performance  appraisal.  In  this  case,  the  lack 
of  a  relationship  between  the  different  criteria  is  as  much  a 
measurement  issue  as  a  conceptual  one.  If  (hypothetically)  the 
learning  measure  assessed  the  same  range  of  behaviors  that  the 
simulation  assessed  (i.e.,  was  more  comprehensive),  we  would 
expect  to  see  a  stronger  relationship  between  them.  Similarly, 
to  the  extent  that  the  transfer  behavior  measure  focused  only  on 
the  trained  behaviors  (i.e.,  was  more  focused),  we  would  expect  a 
greater  correlation  with  training  performance. 

This  is  not  to  suggest  that  training  effectiveness  measures 
should  be  developed  to  ensure  overlapping  content.  In  fact, 
there  are  practical  reasons  why  outcomes  measures  could  focus  on 
different  objectives  (e.g.,  to  ensure  that  the  entire  criterion 
space  is  properly  assessed) .  Furthermore,  organizational 
realities  often  preclude  the  collection  of  "ideal"  training 
effectiveness  measures  (Tannenbaum  &  Woods,  1992).  However,  what 
we  are  suggesting  is  that  there  may  be  a  difference  between  the 
"true"  relationships  among  the  effectiveness  criteria  and  ob¬ 
served  correlations  based  on  the  available  measures  during  a 
given  research  study. 

Overall,  we  hypothesize  positive  relationships  among  most  of 
the  outcomes.  However,  unlike  Hamblin  (1974) ,  we  do  not 
hypothesize  a  link  between  reactions  and  learning.  There  is  no 
logical,  nor  theoretical,  foundation  for  suggesting  that  liking  a 
training  course  should  be  related  to  objective  measures  of 
learning.  In  fact,  a  meta-analysis  of  training  outcomes  revealed 
low  correlations  between  reactions  and  other  training  outcomes  in 
previous  research  (Alliger  &  Janak,  1989) .  Alliger  and  Janak 
suggested  that  negative  correlations  between  reactions  and 
learning  are  possible,  and  that,  "perhaps  it  is  only  when 
trainees  are  challenged  to  the  point  of  experiencing  the  training 
as  somewhat  unpleasant  that  they  learn."  (p.  334). 
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On  the  other  hand,  some  positive  correlations  have  been 
reported  as  well  (e.g.,  Eden  &  Shani,  1982) .  A  possible 
explanation  for  this  inconsistency  is  that  reaction  measures  that 
assess  the  perceived  relevance/utility  of  the  training  are 
positively  related  to  learning,  but  measures  that  assess  a 
trainee's  general  level  of  affect/happiness  are  not.  Baumgartel 
and  Jeanpierre  (1972)  found  positive  correlations  between 
trainees'  perceptions  of  value  and  self-reported  behavior  change, 
but  not  between  hygiene  reaction  measures  and  self-reported 
behavior  change.  Unfortunately,  most  previous  training  studies 
did  not  measure  training  reactions  as  a  multi -dimensional  con¬ 
struct  . 

Since  training  reaction  measures  are  such  a  prevalent  form 
of  training  outcome,  further  research  is  needed  to  clarify  the 
relationship  between  reactions  and  other  measures.  For  example, 
Mathieu  et  al .  (1992)  found  that  reactions  interacted  with 

motivation  to  predict  learning.  It  is  important  to  understand 
the  extent  to  which  reaction  measures  are  likely  to  be  useful 
surrogates  for  other,  more  dif f icult-to-collect ,  training 
effectiveness  indices.  However,  at  this  point,  there  is  little 
evidence  to  suggest  that  reactions  are  related  to  other  training 
outcomes . 

From  a  pragmatic  perspective,  we  agree  with  Goldstein  (1980) 
concerning  the  need  to  use  multiple  criteria  that  reflect 
instructional  objectives  and  organizational  goals.  Each  type  of 
outcome  measure  reveals  something  different  about  the 
effectiveness  of  the  training,  and  thus,  the  appropriate  focus  of 
the  evaluation  should  vary  with  the  situation.  In  addition,  we 
recognize  that  observed  correlations  will  not  always  support  the 
hypothesized  connections  among  training  criteria.  Our  model 
depicts  the  conceptual  relationship  among  the  criteria.  Future 
research  should  examine  the  conditions  under  which  the  measures 
c ovary . 

Now  that  we  have  addressed  the  issue  of  measuring  training 
effectiveness,  we  turn  to  the  other  variables  in  the  model  and 
discuss  their  interrelationships  and  impact  on  training 
effectiveness. 

VARIABLES  IN  THE  TRAINING  EFFECTIVENESS  MODEL 
Individual  Characteristics 

Individual  characteristics  are  hypothesized  to  impact  a 
number  of  variables  throughout  the  model.  First  of  all, 
trainees'  abilities  are  hypothesized  to  influence  learning  and 
training  performance.  Non-ability  factors  (e.g.,  attitudes)  are 
hypothesized  to  influence  trainees'  expectations,  desires,  and 


29 


Technical  Report  93-011 


pre-  and  post -training  motivation.  The  following  sections 
describe,  in  more  detail,  those  individual  characteristics  that 
we  believe  are  important  to  training  effectiveness. 

Ability.  We  use  the  term  "ability"  to  refer  to  a  range  of 
capabilities,  including:  cognitive  ability,  physical  ability, 
task  specific  abilities,  and  trainability . 

1)  Cognitive  Ability.  Many  studies  have  examined  the 
effects  of  cognitive  ability  in  training  environments.  Neel  and 
Dunn  (1959)  found  a  relationship  between  the  Wonder lie  test  and 
course  exam  scores.  Distefano,  Fryer,  and  Crotty  (1988)  studied 
training  for  psychiatric  aides .  They  found  that  two  cognitive 
ability  measures  were  predictive  of  performance  on  a  training 
knowledge  test.  Drakeley,  Harriot,  and  Jones  (1988)  reported  a 
relationship  between  an  intellectual  aptitude  battery  (verbal, 
non-verbal,  numeracy,  speed  and  accuracy)  and  several  training 
effectiveness  measures,  including  professional  marks  (apparently 
a  learning  measure),  and  leadership  ratings.  However,  they  found 
no  relationship  between  the  measure  and  withdrawal  from  training. 

Gladstone  and  Trimmer  (1985)  reported  a  relationship  between 
the  General  Aptitude  Test  Battery  (GATE) ,  and  training  success 
for  work  incentive  program  participants.  Taylor  (1952)  and 
Taylor  and  Tajen  (1948)  reported  relationships  between  aptitude 
test  batteries  and  scores  on  a  training  performance  test.  Gordon 
and  Kleiman  (1976)  reported  an  effect  for  an  intelligence  measure 
and  the  sum  of  graded  test  exercises.  Tubiana  and  Ben-Shakar 
(1962)  noted  the  connection  between  an  intelligence  test  and 
officers'  ratings  of  potential  at  the  conclusion  of  training. 
Mobley,  Hand,  Baker,  and  Meglino  (1979)  found  a  significant 
difference  between  recruit  training  graduates  and  those  that 
failed  to  complete  training  on  the  AFQT  (a  form  of  scoring  the 
Armed  Service  Vocational  Aptitude  Battery  -  ASVAB) . 

Fox,  Taylor,  and  Caylor  (1969)  and  McFann  (1969)  also 
reported  a  relationship  between  the  AFQT  and  training 
effectiveness  as  measured  by  training  time  and  passing  training, 
respectively.  Hogan  and  Hogan  (1985)  found  the  ASVAB  to  be 
unrelated  to  training  completion  for  Navy  divers.  Allen,  Hays, 
and  Buffardi  (1986)  found  that  trainees  scoring  higher  on  the  GRE 
analytic,  and  VPI  intellectual  exams,  took  longer  to  solve 
problems,  but  had  fewer  incorrect  solutions.  In  a  study  of  a 
more  focused  form  of  ability.  Gopher  (1982)  reported  a  positive 
relationship  between  selective  attention  ability  and  completion 
of  a  two-year  training  program  for  Israeli  flight  cadets. 

In  general,  the  research  summarized  here  suggests  that 
trainees  with  greater  ability  will  demonstrate  better  training 
performance  and  higher  scores  on  learning  measures .  In  a  study 
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of  military  trainees,  Ree  and  Earles  (1991)  reported  that  general 
cognitive  ability  ("g")  was  the  best  predictor  of  training 
success.  This  has  important  implications  for  selecting  employees 
for  training,  particularly  if  training  is  costly  and  failure  is 
possible.  However,  these  studies  do  not  allow  us  to  conclude 
that  higher  ability  people  learn  more  in  training. 

Most  of  the  studies  cited  above  that  addressed  "learning, " 
actually  assessed  academic  performance  or  post -training  knowledge 
levels,  and  not  learning,  per  se.  That  is,  the  studies 
demonstrate  that  trainees  who  possess  greater  ability  do  better 
on  performance  and/or  learning  tests  after  training,  but  the 
studies  do  not  indicate  whether  high  ability  individuals  gained 
more  from  training  than  did  low  ability  individuals.  Learning 
implies  a  change  or  an  improvement  in  knowledge  as  a  result  of 
training.  It  is  likely  that  the  higher  ability  people  would  have 
scored  better  on  the  knowledge  tests  even  without  training. 
Schmidt,  Hunter,  and  Outerbridge  (1986)  suggest  that  cognitive 
ability  should  enable  people  to  acquire  job  knowledge.  Until 
recently,  the  premise  that  trainees  with  high  cognitive  ability 
learn  or  acquire  more  knowledge  than  others  during  training  had 
not  received  a  great  deal  of  empirical  attention  in  the  training 
literature . 

In  one  recent  study,  Bretz  and  Thompsett  (1992)  reported  a 
significant  effect  for  cognitive  ability  on  post-training 
knowledge,  after  controlling  for  pre-training  knowledge. 

However,  in  a  similar  sample  in  which  pre-training  knowledge 
could  not  be  controlled  for,  the  relationship  between 
post-training  knowledge  and  cognitive  ability  appeared  to  be  even 
greater.  In  other  words,  cognitive  ability  was  related  to 
learning,  but  was  a  stronger  predictor  of  post -training  knowledge 
than  of  learning,  per  se . 

Over  the  last  several  years,  there  has  been  continuing 
debate  about  the  importance  of  cognitive  ability,  and  whether 
certain  abilities  are  more  important  at  various  points  during 
skill  acquisition  (see  Ackerman,  1989,  1992;  Barrett,  Caldwell,  Sc 
Alexander,  1989;  Fleishman  &  Mumford,  1989a,  b;  Henry  Sc  Hulin, 
1987;  Murphy,  1989) .  For  example,  Murphy  (1989)  suggests  that 
cognitive  ability  is  critical  for  learning  and  performing  new  or 
unfamiliar  tasks,  but  less  critical  during  stages  when  workers 
are  performing  well  learned,  familiar  tasks.  Ackerman  (1988) 
found  that  ability  has  differential  predictability  at  initial, 
intermediate,  and  asymptotic  performance  levels. 

It  seems  logical  that  ability  sets  a  limit  on  learning, 
particularly  for  complex  tasks.  If  training  is  at  a  level  beyond 
a  person's  ability,  no  learning  will  occur.  Perhaps  it  is  best 
to  think  of  ability  as  resource  capacity.  If  sufficient  re- 
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sources  exist,  then  learning  can  occur  and  other  factors  (e.g., 
motivation,  competing  tasks)  will  also  influence  the  degree  of 
learning.  Kanfer  and  Ackerman  (1989)  expanded  on  the  work  of 
Kahneman  (1973)  and  proposed  a  model  of  ability-motivation  inter¬ 
actions.  They  suggest  that  individuals  have  a  particular  re¬ 
source  capacity  level,  and  that  motivational  processes  will 
influence  personal  allocation  of  those  resources.  The  greater 
the  attentional  demands  of  the  task,  the  greater  the  importance 
of  cognitive  ability. 

Future  research  needs  to  assess  the  relative  affects  of 
cognitive  ability  and  motivation  on  pre-post  change  measures  of 
learning  for  various  forms  of  training  tasks.  We  would  speculate 
that  cognitive  ability  measures  are  often  predictive  of  pre-  or 
post-training  learning  measures,  but  that  both  learning 
motivation  and  ability  are  often  predictive  of  pre-post  change. 

Abilities  may  also  interact  with  training  methods  to 
influence  training  effectiveness.  For  example,  Parker  and 
Fleishman  (1961)  showed  that  structuring  training  procedures  to 
match  ability  components  required  at  each  stage  improved 
performance  over  that  seen  in  two  control  groups.  There  is  a  . 
body  of  literature  that  addresses  aptitude  treatment  interactions 
(see  Cronbach  &  Snow,  1977) ;  this  is  discussed  in  the  section  on 
training  methods. 

2)  Physical  Ability.  Two  studies  considered  physical 
ability  as  a  predictor  of  training  effectiveness.  Biersner, 
Ryman,  and  Rahe  (1977)  found  that  the  physical  fitness  of  divers 
was  related  to  their  successful  completion  of  training.  Hogan 
and  Hogan  (1985)  found  cardiovascular  endurance,  lifting 
strength,  and  muscular  endurance  test  scores  predicted  completion 
of  diving  training  and  overall  training  performance. 

Hogan  (1991)  created  several  tables  that  summarized  a  number 
of  physical  fitness  and  ability  tests  relating  to  job  and 
training  performance.  These  tables  demonstrate  that  a  variety  of 
physical  ability  tests  (grip  strength,  cable  pull,  dynamic 
leg/arm  strength,  sit-ups,  step-up  time,  body  density,  balance, 
twist  and  touch) ,  have  been  examined  in  relation  to  training 
performance  (Reilly,  Zedeck,  &  Tenopyr,  1979;  Myers,  Gebhardt, 
Price,  £c  Fleishman,  1981)  . 

In  general,  it  can  be  concluded  from  these,  and  other 
studies,  that  components  of  physical  ability  are  related  to 
aspects  of  training  performance  (e.g.,  time  to  complete 
training) .  In  particular,  these  studies  suggest  that  for 
training  with  a  strong  physical  component,  such  as  underwater 
diver  training,  physical  fitness  is  related  to  training 
performance.  As  with  cognitive  ability,  this  is  important  for 
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selecting  trainees  who  are  likely  to  pass  training.  As  with  any 
selection  paradigm,  it  is  important  to  identify  the  appropriate 
predictors  that  correspond  to  the  performance  criteria. 
Performance  on  training  tasks  with  a  perceptual  ability  component 
are  likely  to  be  predicted  by  perceptual  ability  tests,  and  so 
on.  The  taxonomic  work  of  Fleishman  and  Quaintance  (1984)  is 
informative  in  this  regard. 

3)  Trainability .  Another  group  of  researchers  have  studied 
task-specific  abilities  as  predictors  of  training  performance. 
They  use  performance  on  samples  of  the  task  to  be  trained  or 
early  performance  trials  as  indications  of  trainability. 
"Trainability"  is  the  ability  to  learn  a  given  task.  Robertson 
and  Downs  (1979)  described  the  steps  in  administering  a  train- 
ability  test.  First,  potential  trainees  are  briefly  instructed 
on  how  to  perform  the  task,  and  allowed  to  ask  questions.  Next, 
the  prospect  performs  the  task  unaided  and  is  evaluated  on 
his/her  performance.  In  a  review  of  trainability  studies,  Rob¬ 
ertson  and  Downs  found  that  about  16%  of  the  variance  in  trainee 
performance  is  attributable  to  ability. 

For  the  most  part,  investigations  of  trainability  have 
focused  on  tasks  with  a  manual  component,  and  have  been  conducted 
with  greater  frequency  in  Great  Britain  than  in  the  United 
States.  Several  examples  of  these  studies  are  noted  here.  Downs 
(1970)  reported  that  performance  on  a  training  sample  was  related 
to  final  instructor  ratings.  Gordon  (1955),  and  Gordon  and  Cohen 
(1973)  showed  that  early  training  performance  was  related  to 
subsequent  radio  code  test  secures.  Smith  and  Downs  (1975) 
demonstrated  a  relationship  between  trainability  assessment  and  a 
job  performance  test  three  months  after  training,  although  the 
relationship  diminished  over  12  months.  A  recent  meta-analysis 
showed  a  positive  relationship  between  trainability  test  scores 
and  various  training  and  performance  measures  (Robertson  &  Downs, 
1989) .  However,  in  general,  trainability  tests  predict 
short-term  training  success  better  than  long-term  training 
success,  or  subsequent  job  performance. 

In  sum,  trainability  has  been  shown  to  be  a  useful  predictor 
of  training  and  job  performance,  particularly  for  manual  jobs, 
and  for  short-term  criteria.  As  with  some  of  the  other  predic¬ 
tors,  trainability  measures  are  useful  for  selecting  trainees  in 
situations  where  training  is  costly  or  time  consuming.  However, 
they  do  not  add  much  to  our  conceptual  understanding  of  why 
training  works.  They  tell  us  that  people  who  are  more  capable  of 
learning  a  relevant  portion  of  the  task  will  be  more  capable  of 
learning  the  remainder  of  the  task. 

Self -Efficacy.  As  noted  earlier,  self-efficacy  should  be 
considered  an  important  dependent  variable  because  it  has  been 
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shown  to  be  related  to  subsequent  task  performance  (Barling  & 
Beattie,  1983;  Locke,  Frederick,  Lee,  &  Bobko,  1984;  Taylor  et 
al.,  1984).  Similarly,  pre-training  self-efficacy  may  be  an 
important  predictor  of  learning  and  training  performance.  Re¬ 
cently,  Gist,  Schwoerer,  and  Rosen  (1989)  demonstrated  a 
connection  between  pre-training  self-efficacy  and  subsequent 
training  performance  in  computer  software  training.  Results  of 
this  investigation  revealed  a  Pearson  r  value  =  .31  between  self- 
efficacy  (as  measured  by  a  self-report  questionnaire) ,  and  newly 
trained  software  skills  (as  measured  by  achievement  tests 
performed  on  a  computer  which  utilized  the  new  software) .  On  the 
basis  of  these  results.  Gist,  et  al,  (1989)  suggested  that 
training  benefits  can  be  enhanced  by  first  increasing  self- 
efficacy  via  a  pre-training  intervention  technique.  Along  this 
line,  Eden  and  Ravid  (1982)  manipulated _ trainees'  expectations  of 
their  performance  by  having  a  psychologist  tell  some  military 
trainees  that  they  had  high  success  potential.  They  found  that 
self-expectations  of  performance  were  related  to  subsequent 
trainee  performance. 

Obviously,  the  self-efficacy  construct  holds  promise  as  a 
means  to  improve  our  understanding  of  the  training  effectiveness 
process.  Future  training  research  should  incorporate 
self-efficacy  measures  when  possible.  In  particular,  if  a 
relationship  between  pre-training  self-efficacy  and  measures  of 
training  performance  continues  to  be  shown  (i.e.,  trainees  high 
in  self-efficacy  perform  better) ,  subsequent  research  should 
examine  explicitly  the  organizational/situational  factors  that 
affect  pre-training  self-efficacy.  Moreover,  a  relationship 
between  self-efficacy  and  pre-  and  post-training  motivation  seems 
logical,  although  we  found  no  research  that  addressed  this 
connection  directly. 

Self-efficacy  may  also  apply  to  training  in  other  manners. 
For  instance,  Gist  (1987)  pointed  out  that  low  self-efficacy  may 
indicate  an  area  of  employee  training  needs.  That  is,  an 
employee's  low  self-efficacy  regarding  a  specific  skill  may 
indicate  a  deficiency  in  training.  This  connection  could  be  used 
both  to  plan  future  training,  and  to  evaluate  past  training 
effectiveness  (Gist,  1987) .  In  addition,  pre-training  evaluation 
of  self-efficacy  would  allow  for  the  tailoring  of  training 
programs  to  specific  employees.  For  instance,  when  working  with 
low  efficacy  persons,  utilizing  enactive  mastery  and  modeling 
techniques  could  lead  to  the  most  successful  efficacy 
augmentation  (Gist,  1987) . 

Attitudes.  Trainees'  work  related  attitudes  can  clearly 
affect  their  receptiveness  to  training.  In  particular,  their 
level  of  commitment  to  the  organization  is  likely  to  predispose 
them  to  view  training  as  more  or  less  useful,  both  to  themselves 
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and  to  the  organization.  Organizational  commitment  is  defined  as 
"...the  relative  strength  of  an  individual's  identification  with 
and  involvement  in  a  particular  organization.  Conceptually,  it 
can  be  characterized  by  at  least  three  factors:  (1)  a  strong 
belief  in,  and  acceptance  of,  the  organization's  goals  and 
values;  (2)  a  willingness  to  exert  considerable  effort  on  behalf 
of  the  organization;  and  (3)  a  strong  desire  to  maintain 
membership  in  the  organization"  (Mowday,  Porter,  &  Steers,  1982, 
p.  27)  . 

Accordingly,  it  follows  that  current  employees  who  are  more 
committed  to  the  organization  would  be  more  likely  to:  (1)  per¬ 
ceive  that  training  would  be  beneficial;  (2)  be  willing  to  exert 
a  great  deal  of  effort  in  order  to  be  successful  in  training;  and 
(3)  want  to  do  well  in  training  in  order  to  solidify  their  posi¬ 
tion  in  the  organization.  Furthermore,  Pierce  and  Dunham  (1987) 
have  found  that  new  employees'  propensity  to  become  committed  to 
the  organization  (i.e.,  as  assessed  on  their  first  day  of  employ¬ 
ment)  exhibited  significant  positive  correlations  with  job  and 
role  expectations,  as  well  as  their  willingness  to  take  on  organ¬ 
izational  responsibilities.  Thus,  we  would  expect  that  even  new 
employees'  organizational  commitment  levels  would  be  related 
positively  to  their  expectations  concerning  training,  and  to 
their  motivation  to  learn. 

Very  little  empirical  research  on  commitment  has  been  con¬ 
ducted  within  a  training  context.  Mobley  et  al .  (1979)  found 

that  a  trainee's  intention  to  remain  with  the  military  was 
related  to  completion  of  recruit  training.  Noe  and  Schmitt 
(1986)  found  that  job  involvement  was  related  to  learning,  but 
not  to  behavior  change  or  motivation  to  transfer.  Additional 
research  is  needed  which  examines  the  influence  of  trainees' 
attitudes  on  training  effectiveness. 

Other  trainee  attitudes  or  dispositions  that  might  affect 
training  effectiveness  include  goal  orientation  (Dweck,  1986), 
cognitive  playfulness  (Martocchio  &  Webster,  1992) ,  and  individu¬ 
al  values  about  learning. 

Personality.  Several  studies  have  examined  the  connection 
between  personality  variables  and  training  performance.  In  their 
review  of  individual  differences  in  military  training  environ¬ 
ments,  Hogan,  Arenson,  and  Salas  (1987)  identified  some  instances 
in  which  personality  was  related  to  training  outcomes.  Hogan  and 
Hogan  (1985)  found  that  personality  measures  related  to  good 
adjustment,  risk  taking,  and  confidence  were  correlated  with 
completion  of  training  for  Navy  divers.  Hoskin,  Driskell,  and 
Salas  (1986)  found  that  the  Hogan  Personality  Inventory  accounted 
for  additional  variance  in  Navy  Basic  Electricity  and  Electronics 
School  performance  over  the  ASVAB. 
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Ryman  and  Biersner  (1975)  demonstrated  a  relationship 
between  conformity  and  training  graduation.  Neel  and  Dunn  (1959) 
showed  a  connection  between  the  "how  supervise  scale"  and  an 
authoritarianism  measure  with  training  course  exam  scores. 

Tubiana  and  Ben-Shalchar  (1982)  found  a  composite  measure  of 
personality  and  motivation  to  be  related  to  officer  ratings  of 
trainee  potential.  Unfortunately,  it  is  impossible  to  separate 
the  personality  effects  from  the  motivation  effects  in  that 
study.  Other  personality  traits  that  may  be  relevant  in  the 
training  context  include  openness  to  experience  (Barrick  &  Mount, 
1991) ,  concentration,  persistence,  and  self-confidence  (French, 
1973)  . 

In  contrast,  Baumgartel  and  Jeanpierre  (1972)  found  no  rela¬ 
tionship  between  a  composite  measure  of  personality  and  any 
training  outcome  factors.  Miles  (1965)  reported  no  significant 
relationship  between  ego  strength,  flexibility,  or  need  for_ 
affiliation  with  self-  or  peer-rated  changes.  Noe  and  Schmitt 
(1986)  uncovered  no  connection  between  locus  of  control  and 
measures  of  learning,  motivation  to  transfer,  or  behavior  change. 

Overall,  then,  there  is  only  mixed  support  for  a  direct 
connection  between  personality  and  training  effectiveness. 

Perhaps  stable  personality  traits  operate  through  their  influence 
on  dynamic  trainee  characteristics  (e.g.,  self-efficacy, 
expectations,  or, training  motivation)  to  affect  training 
outcomes.  For  example,  Baumgartel,  Reynolds,  and  Pathan  (1984) 
reported  a  relationship  between  locus  of  control  and  need  to 
achieve  with  self-reported  effort  to  apply  (i.e.,  post-training 
motivation) .  Furthermore,  Mathieu,  Martineau,  and  Tannenbaum 
(1993)  found  that  need  for  achievement  was  directly  related  to 
the  development  of  self-efficacy  during  training  and,  in  turn, 
indirectly  related  to  skill  acquisition.  We  suggest  that  if 
personality  variables  are  likely  to  affect  training 
effectiveness,  their  influence  will  probably  be  indirect,  through 
more  dynamic  trainee  characteristics. 

Experience .  There  is  little  support  for  the  conclusion  that 
experience  directly  influences  training  effectiveness.  Miles 
(1965)  and  Fleishman  (1953)  reported  no  effect  for  tenure  or 
number  of  subordinates  supervised  on  training  effectiveness.  The 
latter  may  be  considered  a  surrogate  measure  of  "type  of 
experience."  Gordon,  Gofer,  and  McCullough  (1986)  found 
seniority  to  be  unrelated  to  time  to  complete  training.  They 
found  previous  job  performance  and  inter- job  similarity  to  be 
related  to  training  completion  time.  However,  those  two 
variables  are  as  easily  considered  measures  of  task-specific 
ability  as  they  are  experiences.  Drakely  et  al.  (1988)  did  find 
a  relationship  between  a  weighted  application  blank  (typically 
they  contain  experiential  information) ,  and  measures  of  learning 
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and  withdrawal.  With  regard  to  experience  with  previous 
training,  Cronbach  and  Snow  (1977)  did  show  effects  for  previous 
experience  with  instructional  techniques. 

In  general,  experience  has  not  been  found  to  be  directly- 
related  to  training  effectiveness,  except  where  it  is  a  surrogate 
measure  of  task-specific  ability.  However,  it  is  likely  that 
experiences  are  useful  predictors  of  training  expectations, 
desires,  motivation,  and  self-efficacy,  although  there  has  been 
no  research  examining  those  relationships. 

r>ftmr»gfT-aphics .  There  is  almost  no  evidence  of  any  consistent 
relationship  between  demographics  and  measures  of  training  effec¬ 
tiveness.  Baumgartel  and  Jeanpierre  (1972)  found  no  effects  for 
education  or  age.  Baumgartel,  Reynolds,  and  Pathan  (1984) 
reported  no  significance  for  rank/ job  level.  Fleishman  (1953) 
found  no  effect  for  age  or  education.  Tubiana  and  Ben-Shakar 
(1982)  reported  a  relationship  between  education  and  ratings  of 
potential.  As  with  personality  and  experience,  if  demographics 
are  likely  to  affect  training  effectiveness,  it  would  be 
indirectly  through  their  relationship  with  expectations  and 
desires. 

Organizational /Situational  Variables 

The  context  in  which  the  training  system  is  embedded  can  be 
a  critical  determinant  of  training  effectiveness.  Organizational 
and  situational  variables  may  influence  variables  in  the  training 
model  both  before  and  after  training.  The  sections  that  follow 
summarize  what  we  believe  are  the  most  crucial  organizational  and 
situational  variables  that  impact  training  effectiveness. 

Pre- training.  Prior  to  training,  organizational  and 
situational  factors  should  have  a  direct  influence  on  training 
expectations,  desires,  and  training  motivation.  Subsequently, 
they  will  have  an  indirect  effect  on  training  effectiveness. 
Organizational  culture,  history,  and  policies  can  shape  trainees' 
expectations  about  training.  For  example,  Eddy,  Glad,  and 
Wilkins  (1967)  found  that  students  from  supportive,  cohesive 
agencies  expressed  higher  degrees  of  interest  in  course  structure 
and  traditional  academic  approaches  to  knowledge  than  those  from 
less  cohesive  and  supportive  agencies.  Weiss  (1978)  found  that 
subordinates  tend  to  adopt  the  work  values  of  their  immediate 
superiors . 

Trainees  look  to  their  work  environment  for  answers  to  many 
questions:  Does  training  matter  in  this  organization?  Does  the 

organization  develop  its  people  and  promote  from  within?  Have 
management  and  labor  had  problems?  Has  training  been  provided  as 
a  punishment  for  poor  performance  or  as  a  reward  for  good  per- 
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formance?  Have  successful  people  in  the  organization  gone  to 
similar  training  courses?  Answers  to  these  questions  could  shape 
employees'  beliefs  about  the  utility  of  training,  including 
perceptions  regarding  the  instrumentality  of  training  for  attain¬ 
ing  desired  outcomes--a  key  component  of  training  motivation. 

For  example,  there  is  some  evidence  that  the  messages 
trainees  receive  can  influence  training  effectiveness. 

Martocchio  (1992)  manipulated  trainee  perceptions  about  the 
usefulness  of  training  in  their  organization  by  providing 
different  instructions  at  the  beginning  of  training.  One  group 
was  informed  that  computer  training  was  an  "opportunity"  and  was 
told  about  the  potential  benefits  and  gains  associated  with  the 
training.  The  second  group  was  provided  neutral  information 
about  the  general  objectives  of  the  training.  All  trainees 
received  identical  training  but  the  group  that  was  lead  to 
believe  that  the  training  was  valuable  demonstrated  greater 
post-training  knowledge  and  efficacy,  and  lower  computer  anxiety 
than  the  group  that  received  a  neutral  message.  Apparently,  the 
signals  trainees  receive  about  training  can  affect  their 
readiness  to  learn. 

1)  Supervisors.  Supervisors  may  be  a  primary  influence  of 
trainee  expectations  and  motivation  through  the  signals  and 
messages  they  send.  The  message  conveyed  by  superiors  may  be 
direct  or  subtle.  In  an  example  of  a  direct  message,  Kaufman 
(1974)  found  that  immediate  supervisors  discouraged  their  subor¬ 
dinates  from  taking  classes  because  it  might  divert  time  and 
effort  away  from  their  job  assignments.  Alternatively,  Huczynski 
and  Louis  (1980)  noted  that  trainees  who  had  a  pre- training 
discussion  with  their  boss  reported  greater  attempts  to  transfer 
what  they  learned.  Similarly,  Cohen  (1990)  found  that  trainees 
with  more  supportive  supervisors  entered  training  with  stronger 
beliefs  that  training  would  be  useful.  Supervisors  can  show 
their  support  for  training  by  helping  employees  establish 
training  goals  (Cohen,  1990) ,  informing  trainees  that  there  will 
be  post-training  follow-up  or  assessment  (Baldwin  Sc  Magjuka,  in 
press),  providing  release  time  to  prepare  for  training,  and 
having  their  work  covered  while  they  are  in  training  (Lee,  1992) . 

2)  Constraints.  Other  cues  in  the  pre-training  environment 

can  also  affect  training  effectiveness.  Mathieu  et  al .  (1992) 

found  that  trainees  who  perceived  many  constraints  on  their 
environment  entered  training  with  lower  motivation  to  learn. 
Apparently  trainees  felt  they  would  not  be  able  to  apply  what 
they  were  about  to  learn,  so  their  belief  in  the  instrumentality 
of  training  was  adversely  effected.  This  suggests  that  one 
method  for  enhancing  training  effectiveness  is  to  identify  and 
eliminate  obstacles  in  the  work  environment  prior  to  conducting 
training. 
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3)  Notification.  The  manner  in  which  trainees  are  selected 
and  notified  of  training  can  also  influence  expectations  and 
motivation.  Baldwin  and  Magjuka  (in  press)  reported  that 
trainees  who  had  received  information  about  the  training  ahead  of 
time,  reported  a  greater  intention  to  apply  what  they  learned 
when  they  returned  to  the  job,  than  trainees  who  received  no 
prior  information.  Alderfer,  Alderfer,  Bell,  and  Jones  (1992) 
found  that  trainees  who  received  more  information  prior  to 
training  had  more  positive  reactions  at  the  conclusion  of  the 
training.  Hicks  and  Klimoski  (1987)  and  Martocchio  (1992)  both 
showed  that  the  nature  of  the  information  provided  to  trainees 
prior  to  training  can  influence  trainee  attitudes.  For  example, 
Hicks  and  Klimoski  (1987)  found  that  trainees  who  received  a 
realistic  description  of  the  training  reported  more  motivation  to 
learn  than  those  who  received  a  traditional,  positive  portrayal 
of  the  training.  However,  they  did  not  find  differences  in 
actual  learning. 

As  noted  earlier,  Martocchio  (1992)  found  that  information 
emphasizing  the  payoffs  associated  with  training  improved 
subsequent  learning.  In  combination,  these  two  studies  suggest 
that  trainees  should  be  provided  with  information  about  the  value 
and  usefulness  of  the  training,  but  only  if  that  information  is 
consistent  with  organizational  reality.  In  addition,  it  may  be 
useful  to  provide  realistic  information  about  the  difficulty,  or 
rigor  of  the  training  to  help  trainees  develop  appropriate  expec¬ 
tations.  In  general,  it  appears  that  providing  trainees  with 
advance  notification  may  be  helpful,  but  it  is  not  clear  whether 
such  notification  enhances  feelings  of  involvement,  creates 
realistic  expectations,  heightens  motivation,  or  allows  time  for 
trainees  to  align  their  personal  goals  with  the  training  goals 
(Tannenbaum  &  Yukl,  1992) . 

4)  Trainee  Choice.  Another  important  contextual  factor  that 
may  influence  training  effectiveness  is  whether  trainees  can 
choose  which  training  they  attend.  In  some  instances,  voluntary 
participation  has  been  shown  to  be  related  to  higher  motivation 
to  learn,  greater  learning,  increased  self-efficacy,  and  more 
positive  trainee  reactions  than  mandatory  attendance  (Cohen, 

1990;  Hicks  &  Klimoski,  1987;  Mathieu,  et  al . ,  1990;  Mathieu  et 
al.,  1993).  In  contrast,  Baldwin  and  Magjuka  (in  press)  found 
that  engineers  who  perceived  training  to  be  mandatory  reported 
greater  intentions  to  apply  what  they  learned  than  did  engineers 
who  viewed  their  attendance  as  voluntary .  Tannenbaum  and  Yukl 
(1992)  speculated  that  when  training  is  not  highly  valued  in  the 
organization,  mandatory  attendance  may  be  demoralizing.  But  when 
trainees'  previous  training  experiences  have  been  positive, 
mandatory  attendance  signals  to  employees  which  training  courses 
are  considered  most  important  by  the  organization. 
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Baldwin,  Magjuka,  and  Loher  (1991)  showed  that  soliciting 
trainees'  input  as  to  which  training  they  want  to  attend  can  en¬ 
hance  motivation  -  if  they  are  allowed  to  attend  their  training 
of  choice.  However,  soliciting  input  can  backfire.  Baldwin  et 
al.  (1991)  found  that  trainees  who  were  asked  to  specify  training 
preferences,  but  were  subsequently  assigned  to  a  different  type 
of  training,  exhibited  lower  motivation  to  learn  than  those 
trainees  who  were  not  asked  for  their  preferences  at  all. 

We  would  also  hypothesize  that  the  purpose  of  the  training 
could  influence  motivation  and  expectations.  Research  is  needed 
that  examines  expectation  and  motivational  differences  between 
trainees  sent  to  training  to  improve  their  current  skills,  to  i 

develop  new  skills  for  their  current  job,  to  certify  their  exist¬ 
ing  skills,  or  to  prepare  for  subsequent  career  moves. 

In  addition,  there  may  be  differences  in  trainee  motivation 
based  on  task  or  job  characteristics.  Employees  training  for 
jobs  with  greater  task  identity  and  significance  may  be  more 
motivated  to  learn  than  those  from  jobs  with  lower  task  charac¬ 
teristics  scores  (e.g. ,  as  based  on  the  Job  Descriptive  Index) . 

Post- training .  After  training,  organizational/situational 
variables  are  hypothesized  to  influence  trainees'  motivation  to 
transfer  what  they  learned,  and  influence  their  subsequent  job 
performance.  Factors  such  as  transfer  climate  and  supervisor 
support  are  hypothesized  to  affect  motivation,  ’  while  resource 
availability  is  hypothesized  to  influence  job  performance 
directly. 

Baumgartel  and  Jeanpierre  (1972),  and  Baumgartel  et  al . 

(1984)  found  that  employees'  perceptions  of  transfer  climate  were 
related  to  effort  to  apply  training.  Trainees  who  reported  that 
their  transfer  environment  had  a  high  appreciation  for 
performance  and  innovation,  encouraged  risk  taking,  and  allowed 
freedom  to  set  goals,  also  reported  greater  effort  to  apply  their 
training. 

Hand,  Richards,  and  Slocum  (1973)  found  that  trainees  who 
perceived  their  organizations  as,  favoring  participation  by 
subordinates,  innovative  behavior,  and  independence  of  thought, 
reported  greater  behavioral  and  attitudinal  changes.  The  effect 
was  non-significant  after  three  months,  but  apparently  grew^ 
stronger  with  time,  and  was  significant  18  months  after  training. 

Miles  (1965)  also  found  that  perceptions  of  the  transfer 
environment  (i.e.,  security,  autonomy,  power,  and  problem  solving 
adequacy)  were  related  to  perceived  change  on  the  job.  Huczynski 
and  Louis  (1980)  found  supervisor  support  (i.e.,  style  and 
attitude)  to  be  the  strongest  predictor  of  self-rated  attempts  at 
transfer.  Trainees  also  reported  that  transfer  was  inhibited  by 
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work  overload,  crisis  work,  and  a  failure  to  convince  older 
workers . 

Several  recent  studies  confirmed  the  importance  of  the  work 
environment  and  improved  upon  earlier  research  by  closely 
examining  transfer  behaviors.  Rouillier  and  Goldstein  (1991) 
hypothesized  that  a  set  of  situational  cues  and  consequences  in 
the  post-training  work  environment  (i.e.,  "transfer  climate"), 
would  contribute  to  positive  behavioral  transfer.  They  examined 
the  effect  of  these  cues  and  consequences  with  a  sample  of 
managers  who  completed  training  and  were  then  randomly  assigned 
to  one  of  102  organizational  units.  Rouillier  and  Goldstein 
found  that,  in  units  with  more  positive  transfer  climate, 
trainees  demonstrated  significantly  more  trained  behaviors,  even 
after  controlling  for  learning  and  unit  performance.  Tracey, 
Tannenbaum,  and  Kavanagh  (1993)  replicated  and  extended  the 
research  by  Rouillier  and  Goldstein.  Consistent  with  Rouillier 
and  Goldstein  (1991) ,  they  showed  that  positive  training  climate 
contributed  to  post -training  behavior,  even  after  controlling  for 
learning  and  pre- training  behavior.  In  addition,  they 
demonstrated  that  managers  who  returned  to  units  that  shared  a 
common  belief  in  the  importance  of  continuous  learning  (i.e.,  a 
"continuous  learning  culture"),  also  demonstrated  better 
behavioral  transfer. 

As  with  the  pre -training  environment,  situational  con¬ 
straints  can  inhibit  transfer  in  the  post-training  environment  as 
well.  Peters  and  O'Connor  (1980)  and  Peters,  O'Connor,  and 
Eulberg  (1985)  examined  a  variety  of  situational  constraints  to 
work  outcomes.  Peters  et  al .  (1985)  conducted  a  literature 
review  of  empirical  studies  and  identified  several  classes  of 
situational  constraints .  While  their  work  did  not  focus  on  a 
training  context  per  se,  their  findings  are  applicable  to  the 
transfer  of  training  situation.  Specifically,  several  con¬ 
straints  should  influence  job  performance  directly,  including: 
poor  time  availability;  shortages  or  inappropriate  resources 
(e.g.,  tools,  equipment,  materials,  supplies);  lack  of  required 
services  from  others;  a  poor  physical  work  environment;  and  a 
lack  of  job  relevant  authority,.  Without  these,  a  trained 
employee  who  wants  to  perform  his/her  job  differently  would  not 
be  able  to  do  so.  Ford  et  al .  (1992)  showed  that,  upon 

completion  of  training,  employees  will  face  differential 
opportunities  to  practice  and  apply  what  they  have  learned. 
Employees  who  receive  no  opportunity  will  not  be  able  to  transfer 
what  they  learned,  and  their  new  skills  will  likely  atrophy  over 
time . 


Alternatively,  situational  constraints  may  not  be  severe 
enough  to  preclude  transfer  entirely,  but  they  might  make  trans¬ 
fer  difficult  enough  to  discourage  the  employee.  Situational 
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constraints  can  reduce  trainees'  perceptions  that  effort  leads  to 
performance,  and  thus,  may  reduce  motivation  to  try.  Schoorman 
and  Schneider's  (1988)  book  on  constraints  and  facilitators 
provides  numerous  illustrations  of  how  situational  factors 
influence  work  effectiveness. 

A  recent  study  has  shown  that  situational  constraints  can 
operate  at  different  levels  to  affect  training  outcomes.  For 
example,  Mathieu  et  al .  (1993)  showed  that  individual  constraints 

on  a  trainee's  schedule  outside  of  training  can  directly  inhibit  . 
the  development  of  self-efficacy  during  training,  foster  negative 
reactions  to  training,  and  indirectly  attenuate  skill 
acquisition.  This  has  important  implications  for  training 
effectiveness  because  it  illustrates  that  activities  outside  of 
training  can  influence  learning  and  reactions  to  training. 
Organizations  that  respond  solely  to  trainee  reactions  may  make 
fruitless  "improvements"  to  a  training  program  if  the  actual 
source  of  discontent  lies  outside  the  training  context  (Mathieu 
et  al. ,  1993)  . 

The  literature  on  situational  constraints  also  supports  the 
research  on  organizational  climate  regarding  the  role  of  supervi¬ 
sor  and  peer  support.  For  example,  Peters  et  al.  (1985)  noted 
problems  with  a  lack  of  support  from  superiors  or  peers  after 
training.  A  trainee  can  return  to  the  job  with  new  skills,  but 
through  a  lack  of  reinforcement,  or  coaching  or  modeling,  may 
lose  his/her  motivation  to  apply  those  new  skills  (Robinson  & 
Robinson,  1985).  On-going,  post-training  feedback  may  be 
-lecessary  to  improve  and  maintain  performance  (Komaki,  Heinzmann, 
&  Lawson,  1980)  .  Michalak  (1981)  reported  that  managers  from 
offices  that  exhibited  on-going  transfer  of  training  met  with 
/their  employees  (i.e.,  the  trainees)  after  training,  and 
announced  changes  as  a  result  of  training.  Managers  from  the 
offices  with  poorer  transfer  used  no  follow-up  procedures  with 
their  returning  employees.  Some  specific  maintenance 
interventions  are  discussed  later  in  this  report. 

Finally,  Morrison  and  Brantner  (1992)  studied  how  well 
people  learn  their  job,  independent  of  training.  In  a  study  of 
600+  mid-level  Navy  officers,  they  found  that  leadership  climate, 
peer  and  subordinate  competence,  and  time  availability  for 
professional  development  were  all  related  to  how  well  the 
officers  learned  their  jobs.  In  summary,  it  appears  that  the 
work  environment  plays  an  important  role  in  creating  a  context  in 
which  individuals  can  learn,  as  well  as  apply,  what  they  have _ 
learned  in  training.  Further  research  is  needed  that  identifies 
and  examines  the  organizational  and  situational  factors  that 
inhibit  or  facilitate  trainees'  use  of  newly  trained  skills. 
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Expectations /Desires 

As  a  result  of  their  individual  characteristics,  as  well  as 
their  previous  experiences,  both  within  and  outside  of  their 
organization,  individuals  enter  training  with  differing  expecta¬ 
tions  and  desires  regarding  training.  These  cognitions  appear  to 
play  a  central  role  in  determining  training  effectiveness. 

Hoiberg  and  Berry  (1978)  reported  that  discharged  military  re¬ 
cruit  trainees  were  more  likely  to  have  expected  training  to 
emphasize  innovative  training  methods,  and  to  minimize  importance 
of  control,  involvement,  and  efficiency,  than  were  successful 
trainees.  The  discharged  trainees  probably  had  unrealistic 
expectations  about  the  recruit  training  environment  (Bourne, 

1967)  . 

Other  researchers  have  also  found  expectations  to  be  related 
to  training  effectiveness.  Ryman  and  Biersner  (1975)  found  that 
course  expectations  were  positively  related  to  graduation,  and 
that  training  concerns  were  negatively  related.  Lefkowitz  (1970) 
found  a  relationship  between  more  realistic  expectations  of 
trainees,  and  subsequent  performance  in  training,  and  on  the  job. 
Hicks  and  Klimoski  (1987)  manipulated  trainees'  expectations 
through  pre-training  notifications.  They  found  that  those  who 
received  realistic  notices  had  greater  motivation  to  learn, 
greater  commitment  to  attend,  and  reported  that  the  workshop  was 
more  appropriate  and  profitable.  Martocchio  (1992)  also 
manipulated  trainee  expectations.  He  found  that  trainee  expecta¬ 
tions  were  related  to  how  much  they  learned  during  training. 

Eden  (1990) ,  in  summarizing  the  research  on  Pygmallion  effects, 
concluded  that  trainee  achievement  can  be  greatly  enhanced  by 
increasing  trainees'  performance  expectations.  On  the  other 
hand,  negative  expectations  regarding  training  can  be  an  obstacle 
to  implementing  re-training  efforts  (National  Association  of 
Broadcasters,  1987). 

In  the  turnover  literature,  expectations  have  been  studied 
from  two  perspectives.  One  deals  with  unrealistic  expectations 
(e.g.,  Wanous,  1977),  and  suggests  that  unrealistic  expectations 
should  be  related  to  dissatisfaction  and  turnover.  Typically, 
lower  expectations  are  assumed  tp  be  more  realistic.  This  ap¬ 
proach  has  only  mixed  support,  and  the  effect  of  realism  on 
turnover  is  weak  at  best  (Louis,  1980) . 

An  alternative  viewpoint  on  expectations  deals  with  unmet 
expectations.  This  approach  operationalizes  unmet  expectations 
as  the  discrepancy  between  initial  expectations  (or  needs/de¬ 
sires)  and  actual  experiences  or  perceptions  (cf.,  Dunnette, 
Arvey,  &  Banas,  1973;  Insel  &  Moos,  1974).  In  several  studies 
(Dunnette,  et  al . ,  1973;  Katzell,  1968;  Ross  &  Zander,  1957), 
dissatisfaction  and  voluntary  turnover  were  related  to  unmet 
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expectations,  and  not  to  initial  expectation  levels.  To  the 
extent  that  higher  expectations  are  unrealistic  and  are 
subsequently  unfulfilled,  the  two  approaches  are  similar. 

However,  when  the  two  approaches  do  not  converge,  the  "met 
expectations"  approach  appears  to  be  more  useful. 

The  turnover  research  has  important  implications  for  train¬ 
ing  effectiveness.  As  noted  earlier,  trainees  enter  training 
with  different  expectations  and  desires.  We  will  use  the  term 
training  fulfillment  to  refer  to  the  extent  to  which  training 
meets  trainees'  expectations  and  desires.  When  training  fails  to 
meet  trainees'  expectations  and  desires,  or  training  fulfillment 
is  low,  we  would  hypothesize  some  dysfunctional  outcomes.  Nega¬ 
tive  attitude  change,  poor  training  reactions,  and  failure  to 
complete  training  could  be  the  results  of  low  training  fulfill¬ 
ment.  In  fact,  Hoiberg  and  Berry  (1978)  found  that  discrepancies 
between  actual  and  expected  training  conditions  accounted  for 
additional  variance  in  Navy  technical  school  graduation,  beyond 
that  accounted  for  by  initial  expectations.  However,  we  found  no 
other  training  research  that  examined  the  effects  of  training 
fulfillment . 

Future  research  should  examine  the  relative  effects  of 
training  fulfillment  on  training  effectiveness  in  conjunction 
with  the  other  relevant  variables  in  the  model.  In  addition, 
"surprisingly  little  attention  has  been  given  to  how  the  point  of 
view  of  employees  relates  to  expectations,  attitudes,  or  deci¬ 
sions  to  select  training  programs"  (Hicks  &  Klimoski,  1987,  p. 
542) .  However,  at  least  one  such  study  has  been  completed  that 
deals  with  self-efficacy  (i.e.,  expectations  an  individual  holds 
regarding  his/her  ability  to  complete  a  task) .  Specifically, 

Hill  et  al.  (1987)  examined  efficacy  expectations  in  relation  to 
decisions  to  use  computers  and  decisions  to  enroll  in  computer 
courses.  The  study  revealed  that  computer  self-efficacy 
predicted  behavior  intentions,  and  behavior  intentions  predicted 
enrollment  in  courses.  People  who  believed  they  could  not 
control  computers  (i.e. ,  had  low  expectations  regarding  their 
potential  performance)  did  not  sign  up  for  the  computer  course. 

Gist  (1987)  noted,  in  this  regard,  that  an  individual  with 
low  self-efficacy,  expecting  not  to  perform  well  in  a  training 
situation,  will  be  prone  to  avoid  the  tpining _ programs .  On  the 
other  hand,  an  individual  with  high  efficacy  will  be  more  likely 
to  voluntarily  attend  training  (Gist  1987) .  There  is  some 
0vidence  of  individual  differences  in  self-reported  training 
needs  (Ford  &  Noe,  1987) ,  but  little  or  no  research  has  examined 
the  antecedents  of  expectations  and  desires  (e.g. ,  organizational 
and/or  situational  factors,  personality,  or  demographics) . 
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Overall,  there  is  a  need  for  research  that  addresses 
questions  such  as:  how  are  training  expectations  formed?;  and 
what  is  the  role  of  co-workers  and  supervisors  in  forming  such 
expectations?  For  example,  Gist  (1987)  recognized  persuasion  as 
a  significant  piece  of  efficacy  information.  Successful 
persuasion  characteristics  include  a  credible  and  expert  source, 
consensus  among  multiple  sources,  and  a  source  familiar  with  task 
demands  (Bandura,  1986) .  Gist  (1987)  proposed  that  a 
supervisor's  high  expectations  might  be  regarded  as  persuasive 
input  to  employee  self-efficacy.  Thus,  a  positive  pre-training 
meeting  may  factor  into  both  increased  self-efficacy  and 
increased  trainee  course  expectation  (Gist,  1987) .  Both  results, 
in  turn,  may  factor  into  increased  training  benefit.  More 
research  in  this  area  is  needed. 

Future  research  should  also  recognize,  explicitly,  that 
training  expectations  and  desires  are  not  necessarily  identical. 
Some  training  expectations  are  negative  (e.g.,  I  expect  that  the 
training  will  require  us  to  complete  peer  assessments,  but  I  hope 
not) .  This  must  be  reflected  in  the  derivation  of  expectation 
fulfillment  indices  as  done  here.  In  this  way,  it  is  possible  to 
reflect  accurately,  not  only  what  a  trainee  expects  to  happen, 
but  also  what  he /she  would  like  to  happen. 

Training  Motivation 

Motivation  refers  to  the  direction  of  attentional  effort, 
the  intensity  of  effort,  and  the  persistence  of  that  effort 
(Kanfer  &  Ackerman,  1989)  .  Training  motivation  is  a  central 
variable  in  our  model  of  training  effectiveness.  It  operates 
throughout  the  model;  before,  during,  and  after  training.  Prior 
to  training,  potential  trainees  may  be  able  to  decide  whether  to 
attend  training,  or  which  training  to  attend.  At  that  point, 
motivation  to  attend  is  crucial.  As  they  enter  training,  and 
during  training,  trainees  will  display  different  degrees  of 
motivation  to  learn.  Finally,  after  training,  for  transfer  to 
occur,  trainees  need  motivation  to  apply  and  maintain  any  new 
skills  they  may  have  acquired  during  training.  Training  motiva¬ 
tion  is  hypothesized  to  influence  learning  directly,  training 
performance  indirectly,  and  to  moderate  the  relationship  between 
training  performance  and  subsequent  job  performance. 

Conceptually,  expectancy  theory  may  provide  a  useful  frame¬ 
work  for  examining  training  motivation  (see  Lawler,  1973  and 
Vroom,  1964,  for  detail  on  expectancy  theory) .  In  the  training 
context,  expectancy  theory  would  suggest  that  trainees  consider 
the  utility  of  the  training  in  attaining  desired  outcomes. 
Trainees  consider  this  in  deciding  whether  to  attend  training,  to 
expend  effort  to  learn,  and  to  persist  in  attempting  to  apply 
what  they  have  learned. 
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More  specifically,  training  motivation  can  incorporate 
several  components.  Trainees  can  consider  whether  their  effort 
in  training  will  lead  to  successful  training  performance,  incor¬ 
porating  questions  such  as:  will  successful  performance  in 
training  lead  to  improvements  in  their  subsequent  job  perform¬ 
ance?;  will  job  performance  yield  certain  outcomes?;  and  how  de¬ 
sirable  are  those  outcomes? 

Trainees'  motivational  focus  may  be  slightly  different  at 
different  points  in  the  process.  Prior  to  training,  trainees  may 
decide  whether  to  participate  in  training,  and  how  much  effort  to 
expend.  The  trainee  may  consider  whether  simply  attending 
training  is  viewed  positively  or  negatively  in  the  organization 
--regardless  of  his/her  effort  and  performance  during  training. 
He/She  may  also  consider  whether  learning,  and  then  applying  the 
training,  will  lead  to  desired  outcomes  down  the  road.  Kanfer 
and  Ackerman  (1989)  would  refer  to  this  as  a  distal  motivational 
process.  It  is  prior  to  task  engagement  and  does  not  draw  atten¬ 
tion  away  from  the  learning  task.  As  noted  earlier,  it  appears 
that  the  manner  in  which  trainees  find  out  about  a  course,  and 
their  degree  of  choice,  can  influence  training  motivation  (Hicks 
Sc  Klimoski,  1987;  Huczynski  &  Louis,  1980)  .  Research  is  needed 
that  further  clarifies  why  trainees  choose  to  attend,  and  what 
influences  their  motivation  to  learn,  prior  to  entering  training. 

During  training,  the  trainees  may  consider  whether  they  can 
learn  the  material  in  the  course.  If  they  feel  they  cannot  learn 
it  (i.e.,  the  link  between  effort  and  learning  or  training  per¬ 
formance  is  zero) ,  they  will  not  be  motivated  to  learn.  This 
link  is  similar  to  the  concept  of  self-efficacy.  Trainees  may 
also  consider  whether  the  material  being  trained  is  relevant  to 
their  job.  If  they  learn  it,  will  it  subsequently  improve  their 
job  performance  (i.e.,  what  is  the  link  between  training  perform¬ 
ance  and  job  performance)? 

After  training,  trainees  will  have  to  decide  whether  or  not 
to  put  effort  into:  (1)  applying  what  they  have  learned,  and  (2) 
continuing  to  use  newly  acquired  skills.  In  addition  to  their 
previous  considerations,  they  are  likely  to  consider  whether 
applying  what  they  learned  will  improve  their  job  performance, 
and  whether  those  improvements  will  lead  to  desired  outcomes 
(e.g.,  promotion,  pay  increase,  recognition).  As  Noe  (1986) 
proposed,  "motivation  to  transfer"  measures  could  include  items 
that  assess  the  trainees'  confidence  in  using  their  new  skills 
and  their  belief  in  the  applicability  of  using  them. _  Naturally, 
a  variety  of  factors  before,  during,  and  after  training  can 
influence  trainee  motivation;  these  are  discussed  throughout  this 
report . 
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Despite  the  centrality  of  motivation  to  most  conceptions  of 
performance,  there  has  not  been  a  great  deal  of  research 
examining  the  role  of  trainee  motivation  in  training 
effectiveness  until  recently.  In  particular,  there  have  been  a 
number  of  studies  that  have  shown  that  training  motivation  is 
related  to  trainee  reactions  (e.g.,  Mathieu  et  al . ,  1992; 
Tannenbaum  et  al . ,  1991),  learning  (e.g.,  Baldwin  et  al . ,  1991; 
Clark,  1990;  Hicks,  1984;  Mathieu  et  al . ,  1992), 
performance/transfer  (e.g.,  Baldwin  et  al . ,  1991;  Facteau, 
Dobbins,  Russell,  Ladd,  &  Kudisch,  1992;  Ralls  &  Klein,  1991) , 
and  completion  of  training  (Biersner  et  al.,  1977;  Mobley  et  al. , 
1979)  . 

In  contrast,  Noe  and  Schmitt  (1986)  found  no  relationship 
between  pre-training  motivation  to  learn,  and  post-training 
learning,  behavior  change,  or  motivation  to  transfer. 
Unfortunately,  a  small  sample  size  and  some  psychometric  problems 
required  them  to  collapse  motivation,  expectation,  and 
situational  variables  together.  Their  resulting  motivation 
measures  are  difficult  to  interpret. 

Measures  of  motivation  based  on  expectancy  theory  have  been 
used  successfully  in  non-training  settings  (e.g.,  Mitchell  & 
Albright,  1972) ,  although  some  concerns  exist  regarding  their 
use.  However,  limited  research  suggests  that  they  may  prove 
useful  in  the  training  context  (Mobley  et  al,  1979;  Mathieu  et 
al.,  1992).  Subscales  could  be  developed  that  focus  on 
motivation  to  attend,  motivation  to  learn,  and  motivation  to _ 
apply.  Alternatively,  an  overall  training  motivation  composite 
measure  could  be  used.  If  so,  it  should  reflect  the  perceived 
utility  of  the  training  to  the  trainee.  It  should  represent  the 
trainees'  perception  that  training  leads  to  valued  outcomes,  and 
stems  from  increases  in  job  performance  attributable  to  training. 

The  recent  research  on  training  motivation  is  encouraging; 
it  is  helping  to  bridge  an  unfortunate  gap  in  our  understanding 
of  training  effectiveness.  Future  research  should  address  the 
measurement  issues  associated  with  training  motivation,  including 
a  consideration  of  longitudinal  data  collection.  In  addition, 
research  is  needed  that  clarifies  the  impact  of  trainee  motiva¬ 
tion  on  training  effectiveness,  both  as  a  main  effect,  and  as  an 
interaction  with  variables  such  as  ability  and  task  complexity. 

Training  Program  Characteristics 

We  do  not  attempt  to  provide  a  comprehensive  review  of  the 
research  related  to  training  factors,  but  instead  will  highlight 
a  few  salient  issues.  As  noted  in  our  model,  a  training  needs 
analysis  should  evaluate  individual,  organizational,  and  task 
factors,  and  should  drive  subsequent  training  design.  To  the 
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extent  that  the  training  needs  analysis  accurately  identifies 
needs,  the  link  between  job  performance  and  results/organization 
al  effectiveness  should  be  strong.  That  is,  if  the  needs 
analysis  is  accurate,  performance  changes  that  occur  due  to 
training  should  contribute  to  organizational  effectiveness.  We 
identified  no  research  that  examined  the  impact  of  various 
methods  of  identifying  training  needs  on  training  effectiveness. 

There  is  a  growing  body  of  research  that  has  examined  the 
effectiveness  of  various  training  methods,  processes,  and 
techniques .  Carroll,  Paine,  and  Ivancevich  (1972)  compared 
training  directors'  ratings  and  empirical  research  findings 
regarding  a  variety  of  training  methods  (e.g.,  business  games, 
case  studies,  programmed  instruction,  lecture)  and  identified 
differences.  A  recent  meta-analysis  of  managerial  training 
(Burke  &  Day,  1986)  found  differences  in  the  effectiveness  for 
different  training  content  (e.g.,  motivation  training,  human 
relations,  decision-making)  and  different  training  methods  (e.g. 
lecture,  behavior  modeling,  group  discussion) .  In  particular, 
behavior  modeling  has  demonstrated  good  success  as  a  training 
method  (Latham  &  Saari,  1979;  Burke  &  Day,  1986;  Gist  et  al . , 
1989;  Baldwin,  1992)  . 

A  variety  of  training  principles  have  been  proposed  and  re¬ 
searched.  Baldwin  and  Ford  (1988) ,  in  their  review  of  transfer 
of  training,  examined  several  training  principles,  including 
sequencing,  practice,  fidelity,  and  the  use  of  identical 
elements.  The  use  of  training  principles  can  influence  training 
effectiveness.  Allen  et  al .  (1986)  reported  significant  effects 

on  training  performance  based  on  simulator  fidelity.  Swezey, 
Perez,  and  Allen  (1988)  found  that  opportunity  for  practice  was 
related  to  training  performance.  Briggs  and  Waters  d958) 
reported  an  effect  for  subtask  or  component  interaction,  and 
Briggs  and  Naylor  (1962)  demonstrated  an  effect  for  part  versus 
whole  task  training.  Nightman  and  Sistrunk  (1987)  reported  a 
chaining  effect  in  their  research.  Kyllonen  and  Alluisi  (1986) 
discuss  a  variety  of  ISD  and  learning  strategy  principles 
designed  to  enhance  training  effectiveness. 

Feedback  has  been  shown  to  improve  performance  (Katzell  & 
Guzzo,  1983) .  Feedback  can  reinforce  positive  performance,  and 
is  necessary  to  correct  negative  performance.  It  can  reveal  to 
trainees  the  gap  between  desired  and  actual  performance,  and  can 
highlight  the  utility  of  training  in  reaching  the  desired  level. 
Thus,  feedback  can  have  a  positive  effect  on  training  motivation 
Komacki,  Heinzemann,  and  Lawson  (1980)  and  Reber  and  Wallin 
(1984)  demonstrated  that  reinforcing  feedback  was  related  to 
subsequent  behavior  in  a  safety  training  research  study.  Miles 
(1965)  found  that  feedback  was  related  to  self-  and  peer-rated 
behavior  change . 
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Bahn  (1973)  and  Kaman  (1985)  have  argued  that  instructor 
characteristics  can  have  an  impact  on  training  effectiveness.  It 
seems  logical  that  instructor  style  and  preparation  can  influence 
trainee  motivation  and  learning.  In  an  interesting  test  of  the 
Pygmalion  effect,  Eden  and  Shani  (1982)  induced  instructors  to 
expect  better  performance  from  some  of  the  trainees.  Results 
indicated  that  the  trainees  for  whom  instructors  were  led  to 
expect  higher  performance  actually  did  perform  better  on 
objective  learning  measures.  Clearly,  the  instructors  were  able 
to  have  an  impact  on  the  learning  of  the  trainees.  Instructors 
may  also  highlight  the  utility  of  learning  to  trainees,  enhancing 
trainees'  perceptions  that  effort  can  lead  to  performance,  and 
that  performance  can  lead  to  desired  outcomes.  (Eden,  1990). 

As  mentioned  earlier,  Eden  (1990)  maintained  that  trainee 
achievement  can  be  enhanced  considerably  by  increasing  trainees' 
performance  expectations.  Adding  this  to  the  above  context, 
instructors  who  expect  trainees  to  perform  well  will  enhance  the 
trainees'  own  expectations  regarding  their  respective  performance 
which,  in  turn,  leads  to  higher  performance. 

Another  factor  to  consider  regarding  training  is  the  recent 
influence  of  cognitive  psychology.  Specifically,  Tannenbaum  and 
Yukl  (1992)  acknowledge  the  growing  cognitive  trend  in  the 
training  field.  As  technological  changes  demand  that  humans  and 
organizations  perform  increasingly  complex  tasks,  the 
significance  and  potential  utility  of  cognitive  learning  models 
will  continue  to  grow  (Tannenbaum  &  Yukl  1992) .  Of  particular 
interest  is  how  humans  acquire  and  maintain  complex,  higher-order 
skills  such  as  problem  solving  and  decision  making. 

One  of  the  implications  of  the  cognitive  psychology 
literature  in  training  is  the  recommendation  to  incorporate 
meta-coanition  skills  into  training.  Meta-cognition  "involves  an 
awareness  of  the  mental  processes  and  strategies  required  for  the 
performance  of  any  cognitive  endeavor."  (Cassidy-Schmitt  &  Newby, 
1986,  p.  29).  In  this  regard,  Dansereau  (1978)  suggested  that 
providing  the  learner  with  a  general  strategy  for  controlling 
intellectual  processing,  that  is,  for  regulating  the  thought 
process  during  performance  will  enhance  effectiveness.  Other 
researchers  believe  that  general  strategic  thinking  cannot  be 
trained  successfully,  and  therefore,  favor  providing 
task-specific  strategies;  the  assumption  being  that  such 
information  will  transfer  to  broader,  but  similar  areas 
(Cassidy-Schmitt  &  Newby,  1986;  Gagne,  1985). 

Kanfer  and  Ackerman  (1989)  have  recently  focused  attention 
on  certain  cognitions  that  can  occur  during  training  (e.g., 
self -regulation,  self -feedback) .  These  cognitions  are  thought  to 
be  part  of  the  proximal  motivation  process,  and  can  determine  the 
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distribution  of  trainees'  effort  to  the  training  task.  For 
example,  whereas  some  learners  intuitively  understand  such  things 
as  the  significance  and  utility  of  trained  skills  or  strategies 
without  explicit  instruction,  this  does  not  appear  to  be  a 
developmental  skill --that  is,  it  does  not  necessarily  increase 
with  age  (Brown,  1980) .  Therefore,  instructors  may  be  able  to 
enhance  trainees'  motivation  to  transfer  acquired  skills  by 
providing  conditional  knowledge,  or  knowledge  of  why  the  new 
skill  is  significant  and  when  it  can  be  used. 

Overall,  there  is  growing  support  for  the  applicability  of 
meta-cognitive  skills  (e.g.,  providing  conditional  knowledge 
during  training)  for  enhancing  the  learning  process 
(Cassidy-Schmitt  &  Newby,  1986) .  This  is  particularly  true  for 
higher-order  tasks,  or  tasks  which  are  stressful  (i.e.,  that 
require  the  trainee  to  perform  in  the  face  of  difficult 
op©3rational  conditions)  (Cannon-Bowers,  Salas  &  Grossman,  1991)  . 

The  mental  model  construct,  also  popular  in  the  cognitive 
psychology  literature  of  late,  may  also  factor  into  training 
system  design.  A  "mental  model,"  as  defined  by  Rouse  and  Morris 
(1986) ,  is  a  "mechanism  whereby  humans  generate  descriptions  of 
system  purpose  and  form,  explanations  of  system  functioning  and 
observed  system  states,  and  predictions  of  future  system  states" 
(p.  360) .  Mental  models  arrange  information  into  structured 
patterns.  This  organization  promotes  rapid  and  flexible 
information  processing  which,  in  turn,  facilitates  access  of 
j^slated  material.  Utilizing  mental  models  allows  classification 
and  retrieval  of  information  about  situations,  objects,  and 
environments  to  be  accomplished  in  terms  of  most  important 
features.  This  technique  is  particularly  beneficial  when 
circumstances  demand  rapid  comprehension  and  response  (Cannon- 
Bowers,  Salas  Sc  Converse,  19  93)  . 

Recently,  Cannon-Bowers  et  al .  (1993)  summarized  the 

implications  of  mental  model  research  for  training  system  design. 
Among  other  things,  these  authors  concluded  that:  (1)  presenting 

explicit  conceptual  models  of  the  system  to  be  trained  can 
enhance  learning;  (2)  training  is  most  effective  when  it  includes 
specific  information  regarding  the  procedures  involved  with  a 
task  (in  addition  to  presentation  of  a  conceptual  model) ;  (3) 

unguided  practice  or  experience  does  not  guarantee  development  of 
complete,  accurate  mental  models  of  the  task;  and  (4)  pre¬ 
existing  mental  models  affect  the  trainee's  ability  to  acquire 
new  knowledge.  In  addition,  developing  a  mental  model  of  the 
overall  system  function,  and  the  role  of  individual  actions  in 
the  system,  can  help  to  demonstrate  the  significance  of 
developing  the  new  skill.  That  is,  a  well  constructed  mental 
model  could  help  trainees  understand  the  impact  of  utilizing 
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newly  acquired  skills  on  both  sub- tasks  or  systems,  as  well  as  on 
the  overall  task  or  systems. 

Several  researchers  have  recognized  the  mental  model 
construct  as  a  valuable  tool  in  evaluating  an  operator's 
knowledge  of  complex  system  performance,  and  as  a  foundation  for 
analyzing  effective  and  ineffective  performance  (Jagacinski  & 
Miller  1978;  Rouse  &  Morris  1986;  Young  1983) .  In  this  regard, 
mental  model  theory  helps  explain  how  operators  can  adapt  to 
changing  conditions,  sequence  task  inputs,  and  recognize  the 
impact  of  a  single  behavior  on  the  overall  system.  Further, 

Rouse  and  Morris  (1986)  argue  that  the  most  worthwhile  use  of  the 
mental  model  construct  may  be  incorporating  it  into  solutions  to 
applied  problems  such  as  developing  training  strategies. 

Several  other  training  factors  are  also  related  to  training 
effectiveness,  and  to  a  trainee's  motivation  to  learn  and 
perform.  For  example,  goal-setting  theory  states  that  an 
individual's  conscious  goals  or  intentions  regulate  his/her 
behavior.  Goal-setting  techniques  can  increase  motivation,  and 
have  demonstrated  increases  in  employee  performance  in  a  variety 
of  settings  (Latham  &  Yukl,  1975) .  Goal-setting  has  been 
incorporated  into  training  with  positive  results  (cf.,  Wexley  & 
Baldwin,  1986;  Wexley  &  Nemeroff,  1975). 

Wexley  and  Latham  (1981)  note  that  the  research  on  goal 
theory  has  three  implications  for  motivating  trainees.  First, 
learning  objectives  should  be  conveyed  clearly  at  the  beginning 
of  training,  and  at  key  points  throughout.  Second,  training 
goals  should  be  difficult  enough  so  that  trainees  are  adequately 
challenged,  but  not  so  difficult  that  they  are  unattainable. 
Third,  the  final  goal,  finishing  training,  should  be  supplemented 
with  periodic  subgoals  throughout  training. 

Goals  set  during  training  that  are  related  to  training 
objectives  may  enhance  learning  and  training  performance.  Goals 
that  are  established  for  the  transfer  environment  may  help  subse¬ 
quent  job  performance.  The  latter  is  discussed  in  the  section  on 
maintenance  interventions. 

Dweck  (1986)  expanded  this  goal  theme  with  a  reexamination 
of  the  affect  of  motivational  processes  on  learning.  Looking  at 
motivation  regarding  achievement,  typical  goals  divide  into  two 
categories:  learning  goals  (increase  competence)  versus 

performance  goals  (receive  approval,  avoid  disapproval)  (Dweck  & 
Elliot,  1983)  .  Dweck  (1986)  divided  motivational  processes  to 
parallel  these  goals.  First,  adaptive  processes  are  attempts  to 
encounter  challenging  learning  situations  that  will  potentially 
increase  personal  skills  and  knowledge.  Conversely,  maladaptive 
processes  are  avoidance  of  challenging  situations  and  preference 
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for  situations  in  which  success  was  previously  achieved  (Dweck, 
1986)  .  Dweck  proposed  that  these  two  different  types  of 
motivation  affect  a  child's  ability  to  exhibit  current  skills  and 
knowledge,  his/her  ability  to  develop  new  skills  and  knowledge, 
and  his/her  ability  to  transfer  the  newly  developed  skills  and 
knowledge  to  unfamiliar  circumstances.  Thus,  motivational 
factors  may  be  a  significant  influence  in  the  use  and  development 
of  natural  aptitudes  (Dweck,  1986) . 

In  another  line  of  thinking,  some  researchers  have  argued 
that  various  training  factors  interact  with  trainee  aptitudes  in 
determining  training  effectiveness.  These  are  referred  to  as 
aptitude-training  interactions  (ATIs) .  For  example,  the  value  of 
practice,  pacing,  format,  and  other  characteristics  may  vary 
according  to  individual  differences.  The  most  thorough 
explication  of  this  concept  is  seen  in  Cronbach  and  Snow  (1977) . 

Several  empirical  examples  of  ATIs  appear  in  the  literature. 
Wightman  and  Sistrunk  (1987)  reported  that  certain  types  of 
training  methods  were  particularly  advantageous  for  low  aptitude 
trainees.  Gist  et  al .  (1989)  noted  a  self-efficacy  by  training 

type  (behavior  modeling  versus  tutorial)  interaction  in  their 
research.  Buffardi  and  Allen  (1986)  found  that  low  ability 
subjects  needed  greater  simulator  fidelity.  That  is,  the  rela¬ 
tionship  between  simulator  fidelity  and  training  performance  was 
moderated  by  analytical  and  mechanical  ability. 

Snow  and  Lohman  (1984)  suggested  that  highly  able  learners 
thrive  on  abstract  instruction,  while  less  able  learners  may  be 
best  trained  by  highly  structured,  concrete  demonstrations. 
Unfortunately,  we  have  not  seen  consistent  ATI  patterns  as  sug¬ 
gested  by  Snow  and  Lohman.  Nonetheless,  we  believe  that  aptitude 
training  interactions  are  still  a  fruitful  area  for  research  and 
may  be  useful  for  better  understanding  why  training  is  effective 
or  ineffective. 

The  training  task  may  also  moderate  the  relationship  between 
variables  in  the  model.  For  example,  task  difficulty,  complexity 
and  type  may  moderate  the  relationship  between  cognitive  ability 
or  motivation  and  training  outcomes.  Kanfer  and  Ackerman  (1989) 
suggest  that,  "the  contribution  of  ability  and  motivation  factors 
to  task  performance  depends  on  the  attentional  demands  imposed  by 
the  task"  (p.  660) .  The  greater  the  attentional  demands  of  the 
training  task,  the  greater  the  importance  of  "G"  or  general 
intelligence . 

Maintenance  Interventions 

Baldwin  and  Ford  (1988)  noted  that  transfer  of  training  to 
the  job  includes  both  the  generalization  of  learned  material  to 
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the  job  as  well  as  the  maintenance  of  trained  skills  over  a 
period  of  time  on  the  job.  Several  researchers  have  developed 
and/or  tested  specific  interventions  designed  to  enhance  the 
transfer  process.  In  our  model,  these  are  hypothesized  to 
heighten  motivation  to  apply  and  maintain  new  skills. 

Marx  (1982)  proposed  a  relapse  prevention  model  to  maintain 
skills  developed  during  managerial  training.  His  approach  con¬ 
sists  of  both  cognitive  and  behavioral  aspects.  It  is  based  on 
identifying  when  relapses  may  occur  (i.e.,  reversions  to  pre¬ 
training  behaviors)  and  developing  strategies  to  deal  with  them. 

Decker  (1982)  incorporated  retention  processes  based  on 
behavior  modeling  training  into  one  training  condition,  and  com¬ 
pared  this  to  one  in  which  trainees  received  the  training  without 
the  retention  processes.  He  found  no  differences  in  training 
reactions.  However,  supervisors  in  the  experimental  condition 
generalized  learning  to  a  novel  situation  better  than  supervisors 
in  the  control  group.  This  replicated  results  from  his  previous 
lab  study  (Decker,  1980) . 

Wexley  and  Baldwin  (1986)  compared  three  strategies  for 
facilitating  positive  transfer  of  training,  including:  (1) 
assigned  goal-setting;  (2)  participative  goal -setting;  and  (3)  a 
behavior  self -management  program  based  on  the  relapse  prevention 
model.  They  found  the  assigned  goal-setting  condition 
demonstrated  greater  learning  than  other  conditions.  Both  goal 
setting  conditions  had  better  self -evaluated  behavior  change  than 
either  the  relapse  prevention,  or  control  conditions.  As  in  our 
discussion  of  training  methods,  it  appears  that  goal-setting  may 
be  useful  in  facilitating  behavior  change. 

The  initial  research  on  specific  post- training  strategies  to 
facilitate  transfer  is  encouraging.  The  importance  of  post- 
training  events  on  training  effectiveness  cannot  be 
overestimated,  and  any  additional  work  that  clarifies  the 
transfer  process  should  prove  extremely  useful . 

LONGITUDINAL  FIELD  INVESTIGATION:  NAVY  RECRUIT  TRAINING 

An  empirical  data  collection  effort  was  conducted  to  begin 
to  test  key  variables  in  the  model  of  training  effectiveness 
shown  in  Figure  1.  Specifically,  we  had  two  purposes  in  mind: 

(1)  to  identify  or  develop  scales  to  measure  key  variables  in  the 
model,  assessing  their  psychometric  qualities  and  providing 
suggestions  for  their  future  use,  and  (2)  to  perform  an  initial 
test  of  key  constructs  and  relationships  from  the  training 
effectiveness  model  in  a  longitudinal  field  training  environment, 
assessing  the  potential  value  of  the  model  for  improving  our 
understanding  of  training  effectiveness. 
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Our  literature  review  identified  a  large  number  of  variables 
that  may  be  related  to  training  effectiveness.  It  was  impossible 
to  design  an  experiment  that  could  measure  all  the  variables  and 
their  interrelationships.  This  can  only  be  accomplished  over 
time,  through  a  programmatic  effort  of  research  with  successive 
studies,  building  on  the  findings  from  previous  studies. 
Therefore,  for  this  investigation,  we  sampled  some  of  the  central 
variables  from  major  categories  within  the  model.  A  variety  of 
individual,  motivation,  expectation,  and  training  effectiveness 
variables  were  included.  Some  of  the  variables  selected  have 
shown  promise  in  the  previous  research,  while  others  have  yet  to 
be  tested  sufficiently.  Most  of  them  have  not  been  tested  in 
such  a  way  as  to  assess  their  relative  impact  on  training 
effectiveness.  The  current  investigation  allowed  for  that 
assessment . 

In  particular,  this  investigation  focused  on  motivation, 
self-efficacy,  and  expectation  variables.  There  have  been  no 
research  studies  that  have  simultaneously  examined  these 
variables  in  such  a  way  as  to  assess  their  relative  impact  on 
training  effectiveness.  The  most  expansive  related  research 
effort  to  date  was  work  by  Noe  and  Schmitt  (1986)  .  Their  study 
provided  stimulation  for  this  research  effort  and  yielded  some 
interesting  and  informative  results.  Unfortunately,  sample  size 
and  psychometric  limitations  forced  them  to  collapse  several 
motivation,  expectation,  and  situational  variables  together. 

This  resulted  in  difficulty  in  interpreting  their  motivational 
variable,  and  precluded  the  examination  cpf  expectations  and 
motivation  separately.  In  addition,  their  study  did  not  include 
self-efficacy  or  ability  measures,  and  was  based  on  a  relatively 
small  sample.  The  present  effort  was  designed  to  overcome  such 
limitations . 
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METHOD 


The  data  collection  was  conducted  at  Recruit  Training 
Command  (RTC)  in  Orlando,  Florida.  This  is  an  eight-week 
training  process  designed  to  train  new  recruits  in  general  Navy- 
procedures.  As  a  longer  (i.e.,  several  weeks  or  more)  training 
program,  it  is  different  than  most  corporate  training  efforts; 
however,  the  U.S.  and  foreign  militaries  use  longer-term  training 
quite  extensively  (e.g..  Gopher,  1982;  Drakely,  Harriot,  &  Jones, 
1988;  Hogan  &  Hogan,  1985) .  Other  long  term  training  efforts  can 
be  seen  for  police  (e.g..  Van  Maanen,  1975),  firefighters, 
psychiatric  aides  (Distefano,  et  al . ,  1988),  stock  brokers,  and 
in  executive  education  programs. 

Subjects  were  1037  trainees  participating  in  recruit 
training.  Their  average  age  was  19.98  (SD  =  2.66).  Of  the  1037 
trainees,  data  were  available  from  666  at  all  three  data 
collection  points.  Average  age  of  the  final  sample  (n  =  666) 
was  19.84  (SD  =  2.43).  The  final  sample  consisted  of  368  men  and 
298  women . 

PROCEDURE 

Within  one  hour  of  their  arrival,  all  trainees  were  asked  to 
complete  a  pre-training  questionnaire  assessing  a  variety  of 
individual  variables,  including  expectations,  attitudes,  self- 
efficacy,  and  pre-training  motivation.  Hoiberg  and  Berry  (1978) 
had  recruits  complete  surveys  within  48  hours  of  their  arrival  at 
bootcamp.  Unfortunately,  during  that  time,  the  recruits  may  have 
changed  their  initial  expectations  based  on  early  training  expe¬ 
riences.  We  chose  the  immediate  administration  of  the  survey  to 
ensure  that  trainee  responses  were  not  based  on  the  training 
experience,  but  were  based  strictly  on  pre- training  factors. 

Participation  was  voluntary  and  no  names  appeared  on  the 
questionnaires.  Social  security  numbers  were  collected  in  order 
to  match  surveys  with  performance  and  cognitive  ability  measures. 
However,  participants  were  assured  of  anonymity  and  no  individual 
responses  were  revealed.  The  ASVAB,  a  measure  of  cognitive 
ability,  was  collected  as  part  of  the  enlistment  process,  prior 
to  recruit  training. 

During  training,  recruits  were  involved  in  classroom  and_ 
field  learning  experiences,  and  in  addition,  completed  academic 
and  physical  tests,  received  numerous  inspections,  and  received 
honors  and  demerits  indicative  of  their  performance  during 
training.  At  the  conclusion  of  training,  trainees  completed  a 
post-training  questionnaire  that  assessed  post-training 
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motivation,  attitudes,  training  perceptions,  self-evaluations, 
and  training  reactions.  Table  2  shows  the  research  design,  and 
illustrates  when  each  variable  was  measured. 

Pre-training  questionnaires  were  completed  by  932  trainees, 
post-training  questionnaires  by  753  trainees  (some  of  whom  had 
not  received  the  pre-training  questionnaire) ,  and  "hard  card" 


Table  2 

Research  Design;  Variables  and  Time  of  Measurement 


TIME  OF  MEASUREMENT 

Pre -Training 

During  Training 

Post- Training 

Cognitive  Ability 

Academic  Tests 

Attitudes 

Attitudes 

Honors 

Self-Efficacy 

Self-Efficacy 

Demerits 

Perceptions  of 
Training 

Performance 

Inspection 

Expectations 

Scores 

Motivation 

Training 

Expectations 

Reactions 

Self-Rated 

Pre-Training 

Motivation 

Performance 

Training 

Fulfillment 

data  (i.e.,  archival  training  performance  and  cognitive  ability 
measures)  were  available  for  855  of  the  trainees.  This  resulted 
in  the  final  sample  of  666  subjects,  from  whom  data  had  been  col¬ 
lected  at  all  three  times. 

Since  the  primary  tests  of  the  model  require  training 
performance  and  post-training  data,  the  majority  of  analyses  were 
conducted  on  a  final  sample  that  consisted  only  of  those  trainees 
who  completed  training.  The  exception  to  this  was  an  analysis  to 
examine  factors  that  influenced  attrition,  which  included  samples 
of  recruits  that  did  and  did  not  complete  the  training. 
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MEASUREMENT  SCALES 

Except  where  otherwise  noted,  all  measures  were  based  on 
seven  point  Likert-type  scales,  ranging  from  'Strongly  Disagree' 
(1)  to  'Strongly  Agree'  (7)  with  'Neither  Agree  nor  Disagree  (4) 
as  the  midpoint.  Some  items  were  negatively  worded,  and  reverse 
coded  in  later  analyses.  As  part  of  the  development  process,  the 
surveys  were  pilot  tested  with  a  small  sample  of  recruits  to 
ensure  clarity  of  wording  and  instructions. 

Some  of  the  measures  represented  existing  scales,  while 
others  were  developed  specifically  for  this  investigation.  All 
new  scales  were  subject  to  factor  analysis  (principle  axis 
utilizing  oblique  [Oblimin]  rotation)  to  assess  their  factor 
structure  based  on  the  total  sample  (N  =  1037) .  Initially,  the 
number  of  factors  was  determined  based  on  eight  values  of  greater 
than  1.0.  The  resulting  factor  structure  was  examined  for 
clarity  of  interpretation  (i.e.,  items  with  high  factor  loadings 
on  only  one  factor  and  conceptual  similarity  of  items  that  loaded 
on  the  same  factor) . 

In  some  instances,  the  initial  solution  demonstrated  complex 
loadings  (i.e.,  items  demonstrating  factor  loadings  of  greater 
than  .40  on  two  or  more  factors)  or  items  that  failed  to  load  on 
any  factor  (i.e.,  no  factor  loading  greater  than  .40  on  any 
factor) .  In  these  cases,  inter- item  correlations  were  examined 
and  additional  factor  analyses  were  conducted  to  establish  a 
structure  with  the  best  fit  both  psychometrically  and  conceptual¬ 
ly- 


Finally,  Cronbach  alphas  were  computed  to  assess  scale 
reliability.  For  uniformity,  Cronbach  alphas  were  computed  for 
all  multi-item  scales  based  on  the  final  sample  of  666  recruits. 

On  the  basis  of  these  analyses,  some  items  were  dropped  and 
some  scales  revised.  Table  3  presents  sample  items  for  most  of 
the  scales.  Table  4  reports  the  number  of  items  in  each  scale, 
scale  means,  standard  deviations,  and  Cronbach  alphas.  Scores 
were  computed  for  each  multi-item  scale  by  averaging  the  items 
each  respondent  completed. 

Below  we  describe  each  of  the  scales,  including  the 
rationale  for  revisions  when  appropriate. 
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Table  3 

Sample  Items  from  Measurement  Scales 


"I  have  excellent  reflexes"  (physical  self-efficacy) 

"I  do  well  in  activities  where  I  have  to  remember  lots  of  information"  (academic  self-efficacvl 

"How  well  do  you  think  you  will  perform  on  room  inspections  at  Recruit  Training"  (training 
performance  expectations) 

"I  expect  that  the  training  will  provide  intense  controls  over  my  behavioj"  (training  expectations) 

"I  hope  that  the  material  I  learn  will  be  presented  to  me  in  lecture  format"  (training  desires) 

"Being  in  the  Navy  is  important  to  me"  (commitment) 

"I  plan  to  re-enlist  after  my  first  tour"  (intent  to  remain) 

"If  1  am  successful  in  recruit  training  it  will  better  enable  me  to  perform  my  job  in  the  Navy"  (training 
to  performance  link  -  training  motivation) 

"If  I  learn  to  perform  well  as  a  result  of  recruit  training  it  will  help  me  to  get  promoted  faster" 
(performance  to  outcome  link  -  training  motivation) 

"Getting  good  duty  stations  and  assignments  are  important  to  me"  (valence  -  training  motivation) 

"1  had  a  chance  to  practice  what  1  learned"  (training  perceptions) 

"I  learned  new  skills  and  knowledge  at  Recruit  Training"  (relevance/value-training  reactions) 

"I  have  been  happy  at  RTC"  (affect/happiness-training  reactions) 
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Table  4 


Srale  Means.  Standard  Deviations,  and  Cronbach  Alohas* 


SCALE 

#  OF  ITEMS 

MEAN 

SD 

ALPHA 

INDIVIDUAL  VARIABLES 

(PRE-TRAINING) 

Ability 

Cognitive  Ability  (ASVAB) 

64.06 

21.23 

N/A 

Attitudes 

Intent  to  Remain 

2 

4.67 

1.34 

.91 

Commitment 

n 

5.89 

.69 

.82 

Self-Efficacv 

Physical  Self  Efficacy  (PSE) 

10 

4.88 

.88 

.85 

Academic  Self  Efficacy  (ASE) 

8 

5.27 

.99 

.87 

Demographics 

Sex  (1  =female,  2=male) 

1 

1.55 

.50 

N/A 

Age 

1 

19.84 

2.43 

N/A 

Family  History  (#  of  military 

1 

2.06 

1.70 

N/A 

relatives) 

EXPECTATION/DESIRE  VARIABLES 

Traininc  Expectations 

Overall 

15 

6.26 

.54 

.82 

Controlled  Learning  Environment 

6 

6.53 

.57 

.80 

Challenge 

3 

6.50 

.74 

.76 

Interactions  With  Compan\' 

4 

6.46 

.68 

.83 

Members 

Training  Method 

2 

4.69 

1.43 

.46 

Training  Desires*' 

Overall 

15 

2.16 

.62 

.84 

Controlled  Learning  Environment 

6 

2.38 

.69 

.83 

Challenge 

3 

1.71 

1.12 

.84 

Interactions  With  Compam’ 

4 

2.70 

.55 

.86 

Members 

Training  Method 

2 

1.12 

1.48 

.53 

Training  Performance  Expectations 

Training  Performance  Expectations 

5 

5.43 

.81 

.83 

MOTIVATION  VARIABLES 

Pre-Training  Motivation 

Instrumentalities 

12 

6.22 

.71 

.89 

Training  Performance 

6 

6.56 

,68 

.92 

Expectancies 

Valence 

12 

6.42 

,60 

.88 

Pre-Training  Motivation 

12 

268.12 

65.25 

.96 

Post-Training  Motivation 

Instrumentalities 

12 

6.07 

.90 

.90 

Training  Performance  Expectancies 

6 

6.37 

.90 

.95 

Valence 

12 

6.48 

.58 

.86 

Post-Training  Motivation" 

12 

258.97 

74.41 

.97 
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Table  4  (continued) 


SCALE 

if  OF  ITEMS 

MEAN 

SD 

ALPHA 

EXPECTATION  FULFILLMENT 

VARIABLES 

Expectation  Fulfillmenf* 

-1.12 

1.48 

Perceptions  of  Training 

5.89 

.69 

INDIVIDUAL  VARIABLES 

(POST-TRAINING) 

... 

Attitudes 

■■ 

Intent  to  Remain 

2 

1.57 

.89 

Commitment 

11 

.76 

.83 

Physical  Self-Efficacy  (PSE) 

10 

.90 

.87 

Academic  Self-Efficacy  (ASE) 

8 

.94 

.87 

TRAINING  EFFECTIVENESS 

VARIABLES 

Training  Reactions 

Overall 

12 

5.98 

.88 

,91 

RelevanceA^alue 

7 

6.28 

,83 

.88 

Happiness 

2 

5.10 

1.46 

.84 

Training  Performance 

Academic  Test  Performance 

4 

325.42 

27.48 

,84 

Honors  (2  =  yes;  1  =no) 

1 

1.06 

.24 

N/A 

Demerits 

1 

4.15 

2.23 

N/A 

Inspections  (1  =  unsatisfactory: 

21 

1.81 

.10 

.52 

2  =  satisfactor\') 

Demerits  and  Inspections  (z  score) 

_ 

.04 

.79 

N/A 

Overall  Performance- 
Combined  Score  (z  score) 

Self-Rated  Physical  Test 

.01 

.57 

N/A 

Performance 

Self-Rated  Overall  Training 

1 

5.90 

1.13 

N/A 

Performance 

5 

5.19 

.86 

.69 

“Sample  sizes  range  from  651  to  666  due  to  missing  responses. 
‘’Scale  can  range  from  -3  to  +3. 

"Scale  can  range  from  1  to  343. 

'‘Scale  can  range  from  -18  to  -1-18. 
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Individual  Variables 

Cognitive  Ability.  Cognitive  ability  was  assessed  using  the 
ASVAB  based  on  AFQT  scoring.  Higher  scores  are  indicative  of 
greater  cognitive  ability. 

Attitudes.  Two  trainee  attitudes  were  measured,  commitment 
and  intent  to  remain.  They  were  measured  prior  to  training,  and 
again  at  the  completion  of  training. 

"Organizational  commitment"  was  assessed  using  11  items 
adapted  from  Mowday  et  al . ' s  (1982)  15  item  scale.  The  full 
length  scale  has  demonstrated  high  reliabilities  in  previous 
research  (average  alpha  =  .88  across  80  samples  with  a  total  N  of 
over  24,000;  see  Mathieu  &  Zajac,  1990).  The  11  items  used  in 
the  present  study  were  selected  based  on  their  relevance  to  the 
Navy  training  environment,  and  demonstrated  -  sufficient  alphas  at 
both  administrations  (i.e.,  .82  and  .83). 

"Intent  to  remain"  was  a  two  item  scale  loosely  based  on 
Martin's  (1979)  work.  Martin  used  two  items,  one  referring  to 
intentions  to  remain  with  the  organization  within  the  next  year, 
and  one  referring  to  longer-term  career  plans.  Since  recruits  do 
not  have  a  true  decision  point  within  the  next  year,  we  revised 
our  scale  accordingly.  Thus,  the  two  items  we  used  were:  "I 
plan  to  re-enlist  after  my  first  tour"  and  "Iplan  to  make  a 
career  out  of  the  Navy."  Alphas  were  .91  and  .89  for  the  pre- 
and  post-training  administrations. 

Self-efficacv  Measures.  These  scales  were  based  on  the  work 
of  Mclntire  and  Levine  (1984) .  Their  scales  were  originally 
designed  for  a  college  population  so  some  rewording  was  neces¬ 
sary.  "Academic  Self-Efficacy"  assessed  trainees'  beliefs  in 
their  ability  to  accomplish  academic  tasks.  A  factor  analysis  of 
the  10  academic  self-efficacy  items  yielded  a  three  factor 
solution.  However,  neither  this  solution,  nor  forced  two  or  four 
factor  solutions,  produced  interpretable  results.  A  close 
examination  of  inter-item  correlations  and  item  content  suggested 
that  subjects  were  having  a  problem  with  the  two  negatively 
worded  items.  These  two  items  were  dropped,  yielding  an  eight 
item  scale  with  acceptable  alphas  at  both  administrations. 

"Physical  Self-Efficacy"  measured  perceived  competence  on 
physical  tasks.  A  factor  analysis  of  the  10  items  relating  to 
physical  self-efficacy  yielded  a  single  factor  solution.  The 
scale  demonstrated  acceptable  alphas  at  both  administrations. 

Demographics .  Gender,  age,  and  a  list  of  family  memlDers  who 
served  in  the  military  were  collected  at  the  start  of  training. 
"Family  history"  refers  to  the  number  of  family  members  who. 
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served  in  the  military.  A  higher  number  indicates  a  stronger 
family  history  with  the  military.  Family  members  who  have  served 
in  the  military  are  a  potential  source  of  information  regarding 
recruit  training  and  the  military  experience.  Family  history 
could  be  related  to  training  expectations  and  desires. 

Expectation/Desire  Variables 

Training  Expectations.  As  noted,  training  expectations, 
desires,  and  perceptions  must  be  tailored  to  fit  the  specific 
training  environment.  Potential  items  were  identified  in  Hoiberg 
and  Berry  (1978)  and  Noe  and  Schmitt  (1986),  and  supplemented  by 
items  we  developed.  Next,  meetings  with  subject  matter  experts 
helped  delineate  the  issues  of  relevance  to  new  recruits,  and  the 
final  list  of  16  training  expectation  items  (and  parallel  desires 
and  perception  items)  were  selected  for  inclusion  in  the  current 
study. 

A  factor  analysis  was  conducted  on  the  16  training 
expectation  items.  All  but  one  item  loaded  cleanly  on  a  four 
factor  solution  (oblique  rotation) .  This  item  did  not  load  in  a 
forced  three  factor  solution  either,  and  the  overall  factor 
structure  was  less  interpretable  than  the  four  factor  solution. 

An  examination  of  inter- item  correlations  confirmed  the  need  to 
drop  the  item  from  the  scale. 

After  dropping  the  item,  the  four  factor  solution  was 
interpretable.  The  four  factors  were  labeled:  "expectations  - 
controlled  learning  environment";  "expectations  -  challenge"; 
"expectations  -  interactions  with  others";  and  "expectations  - 
training  method" . 

Alphas  were  acceptable  for  all  subscales,  except  training 
method,  which  demonstrated  an  alpha  of  .46.  It  would  have  been 
interesting  to  be  able  to  examine  subsets  of  expectations,  but 
unfortunately,  the  low  alpha  precluded  the  use  of  subscales.  A 
total  expectations  scale  was  also  formed  (labeled  "training 
expectations")  from  items  1  through  15.  This  scale  demonstrated 
an  acceptable  alpha  level  and  was  used  in  subsequent  analyses.  A 
higher  score  on  this  scale  indicates  that,  overall,  the  trainee 
has  greater  expectations  of  training. 

Some  additional  work  is  needed  on  these  scales,  including 
the  introduction  of  additional  training  method-type  items. 
Expectation  items  will  likely  need  to  be  generated  .for  each 
unique  training  situation  (clearly  some  of  the  items  used  in  this 
investigation  would  be  inappropriate  for  examining  corporate 
management  training) .  Nevertheless,  the  initial  factor  analyses 
and  psychometric  assessments  are  somewhat  encouraging,  and 
further  work  with  this  construct  is  warranted. 
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Training  Desires .  Training  desire  items  parallel  the  train¬ 
ing  expectation  items.  That  is,  for  each  training  expectation 
item  ("I  expect  that..."),  a  similar  question  was  developed  that 
assessed  desirability  ("I  hope  that...")  .  Factor  analyses  and 

Cronbach  alpha  computations  demonstrated  similar  patterns  as  the 
training  expectation  items,  and  were  treated  in  the  same  fashion. 

The  desire  items  were  recoded  from  a  1  to  7  scale  into  a  -3 
to  +3  scale.  This  was  done  to  support  the  computation  of  the 
training  fulfillment  scale,  and  is  discussed  in  that  section.  A 
higher  score  on  the  training  desire  scale  indicates  that  the 
trainee  hoped  for  more  from  the  training. 

Training  Performance  Expectations.  Five  items  were  written 
that  asked  trainees  "how  well  do  you  think  you  will  perform  on. . . 
[each  major  performance  factor]."  The  five  items  addressed 
physical  performance  tests,  academic  tests,  uniform  inspections, 
room  inspections,  and  overall  performance.  These  items  were 
based  on  a  seven  point  Likert -type  scale  ranging  from  "Not  at  all 
Well"  (1)  to  "Extremely  Well"  (7) ,  with  "Moderately  Well"  (4)  as 
the  midpoint . 

Factor  analysis  yielded  a  clean  one  factor  solution  and  the 
scale  alpha  was  acceptable. 

Training  Motivation  Variables 

Training  motivation  was  assessed  using  a  Valence- 
Instrumentality-Expectancy  approach  (cf.,  Vroom,  1964;  Lawler, 
1973).  Specifically,  trainees'  perceptions  of  the  relationship 
between  performance  in  training  and  future  job  performance  were 
assessed  using  a  six  item  scale.  An  example  item  is  "If  I  am 
successful  in  recruit  training  it  will  better  enable  me  to 
perform  my  job  in  the  Navy."  The  average  of  trainees'  responses 
to  these  items  will  be  referred  to  as  training-performance 
expectancies . 

Trainees  also  provided  ratings  of  the  extent  to  which  they 
perceived  that  higher  performance  in  Navy  jobs  would  lead  to  a 
set  of  12  outcomes  (hereafter  referred  to  as  instrumentalities) . 
These  outcomes  include  such  things  as:  money,  prestige,  respect 
from  family  and  friends,  and  an  opportunity  to  serve  the  country. 
Finally,  trainees'  provided  separate  ratings  of  the  importance  of 
each  of  the  12  outcomes  (hereafter  referred  to  as  valences) . 
Although  the  instrumentalities  and  valences  were  responded  to  on 
a  1-7  point  scale,  the  scale  anchors  were  recoded  to  a  -3  to  +3 
range  to  reflect  both  positive  and  negative  values.  This 
recoding  was  necessary  in  order  to  maintain  the  motivating 
direction  of  combining  instrumentality  and  valence  scores  of 
different  signs  (cf.,  Mathieu,  1987). 
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Training  motivation  scores  were  calculated  by  first 
multiplying  each  outcome's  instrumentality  times  its  valence,  and 
then  multiplying  the  product  by  the  trainees'  training- 
performance  expectancy  score.  This  process  yields  12  composite 
scores,  each  reflecting  a  perceived  motivation  consequence  of 
performing  well  in  training..  Combining  the  12  composites  yields 
a  total  training  motivation  score.  The  combined  scale  exhibited 
high  reliabilities  at  both  times. 

Expectation  Fulfillment  Variables 

We  use  the  term  "Expectation  Fulfillment"  to  refer  to  the 
extent  to  which  training  met  trainees'  expectations  and  desires. 
Different  trainees  may  have  the  same  expectations  (e.g.,  training 
will  be  challenging)  but  one  could  desire  it  (i.e.,  desires 
challenge)  and  the  other  might  not  (i.e.,  desires  easy  training). 
Therefore,  it  is  insufficient  to  rely  simply  on  a  comparison  of 
pre-training  expectations  with  post-training  perceptions  of 
training.  In  our  example,  the  first  trainee  should  be  pleased  if 
training  was  challenging,  but  the  second  would  likely  be 
displeased . 

In  addition,  a  trainee  might  desire  something  but  realize 
that  it  is  not  likely  to  happen  (as  reflected  by  low 
expectations) .  In  this  case,  the  trainee  would  be  pleasantly 
surprised  if  his/her  desire  was  met.  However,  if  the  training 
did  not  fulfill  his/her  desires,  we  would  not  -expect  him/her  to 
demonstrate  the  same  level  of  disappointment  as  if  he/she  both 
desired  and  expected  fulfillment. 

For  these  reasons,  we  developed  an  index  based  on  all  three 
elements:  expectations,  desires,  and  perceptions  of  training. 

We  contemplated  using  a  simpler  measure,  asking  trainees  if 
training  met  their  expectations.  However,  this  form  of 
retrospective  query  is  subject  to  cognitive  distortions. 

Trainees  who  feel  good  may  report  that  training  met  their 
expectations  regardless  of  what  they  desired  or  expected  before 
training.  Simply  asking  trainees  for  their  perceptions  of 
training,  as  we  did  (e.g. ,  did  you  have  a  chance  to  practice^ 
during  training?) ,  removes  some  of  the  subjectivity.  Computing 
an  index  based  on  pre-training  desires  and  expectations,  in 
comparison  to  subsequent  perceptions  of  training,  should  yield  a 
more  objective  measure  of  the  extent  to  which  training  fulfilled 
expectations  and  desires. 

Perceptions  of  Training.  Sixteen  items  were  written  that 
parallel  the  expectation  and  desirability  items,  and  assessed 
trainees'  perceptions  of  the  training  (e.g.,  training  emphasized 
efficiency,  attention  to  detail,  and  chance  to  practice,  and  was 
mentally  challenging) .  This  was  collapsed  into  a  fifteen  item 
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scale,  and  four  subscales,  in  identical  fashion  as  the  training 
expectations  and  desires  scales. 

Expectation  Fulfillment.  Expectation  fulfillment  was 
computed  as  a  function  of  expectations,  desires,  and  perceptions: 


Ef='£=(PrEi)Di 


i=l 


where  Ef  =  total  expectation  fulfillment  score;  i  =  item;  j  =  the 
number  of  perception-expectation  item  pairs;  P  =  perceptions 
(ranges  from  1  to  7) ;  E  =  expectations  (ranges  from  1  to  7) ;  and 
D  =  desires  (ranges  from  -3  to  +3) ,  yielding  an  expectation 
fulfillment  score  that  could  range  from  -18  to  +18. 

The  recoding  of  desires  from  1  to  7  to  -3  to  +3  was  designed 
to  address  one  of  the  issues  noted  above;  perceptions  exceeding 
expectations  have  different  meanings  based  on  desirability. 

Thus,  perceptions  of  challenge  exceeding  expectations  is  a  posi¬ 
tive  experience  for  someone  who  desires  challenge,  a  negative  one 
for  someone  who  does  not.  This  is  incorporated  into  our 
computational  formula.  Similarly,  less  challenge  than  expected 
is  positive  for  someone  who  did  not  desire  challenge,  but 
negative  for  someone  who  desired  challenge.  If  he/she  did  not 
care  about  that  facet  of  training  (zero  on  desirability  scale) , 
his/her  score  on  that  training  fulfillment  item  would  be  zero 
regardless  of  his/her  expectations  or  perceptions,  by  virtue  of 
the  multiplicative  function  in  the  equation. 

The  expectation  fulfillment  scale  was  computed  by  summing 
the  computations  from  each  expectation-desire-perception  triad. 

A  higher  score  is  indicative  of  greater  levels  of  fulfillment. 

The  reliability  estimates  for  the  expectation  fulfillment  scale 
were  found  to  be  at  an  acceptable  level.  However,  the  use  of 
expectation  fulfillment  subscales  is  impossible  until  further 
refinements  of  the  expectation,  desire,  and  perception  scales 
yield  more  reliable  subscales  at  that  level. 

Training  Reactions 

Twelve  training  reaction  items  were  written  to  assess 
various  components  of  trainees'  reactions  to  training.  Some 
items  were  designed  to  tap  into  relevance/perceived  value,  while 
others  were  more  affective  in  nature. 

A  two  factor  analysis  of  the  12  items,  rotated  to  an  oblique 
solution,  did  not  yield  parsimonious  results.  This  two  factor 
solution  had  three  items  with  complex  loadings.  These  items  were 
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dropped  and  another  factor  analysis  was  conducted.  This  yielded 
a  clean  two  factor  solution  with  7  items  loading  on  factor  I 
(labeled  "training  reactions  -  relevance/value")  and  2  items 
loading  on  factor  II  (labeled  "training  reactions  -  affect/happi¬ 
ness").  This  solution  is  consistent  with  the  earlier  discussion 
of  training  reactions  as  a  multi -dimensional  construct.  The  two 
scales  are  correlated,  but  do  not  demonstrate  multicollinearity 
(r  =  . 51 ,  p  <  . 0001) . 

It  should  be  noted  that  when  all  12  items  were  forced  into  a 
single  factor  solution,  they  all  demonstrated  factor  loadings 
greater  than  .40.  In  order  to  parallel  previous  studies  that  did 
not  differentiate  dimensions  of  training  reactions,  we  provide 
some  information  on  the  full  twelve  item  scale  (labeled  "training 
reactions  -  overall"),  although  the  two  factor  solution  provides 
more  detail. 

Training  Performance 

Measures  of  learning  or  training  performance  were  limited  to 
those  currently  used  in  the  RTC  environment.  Although  academic 
tests  were  administered,  there  was  no  true  measure  of  learning 
possible,  since  no  pre-tests  were  conducted.  As  such,  all  the 
measures  are  discussed  under  the  heading  of  training  performance, 
although  they  do  not  fit  neatly  into  the  categories  identified  in 
the  review. 

Correlations  were  computed  among  the  training  performance 
and  reaction  measures  to  see  if  any  of  the  measures  were 
measuring  similar  constructs,  and  to  determine  if  a  single 
training  performance  measure  was  possible.  Table  5  shows  the 
correlations.  Only  demerits  and  honors  were  so  highly  correlated 
as  to  require  collapsing  into  a  single  composite  score  (r  =  -.74, 
p  <  .0001) .  Each  measure  is  discussed  below. 

Demerits .  Demerits  are  assigned  to  recruits  for  poor 
behavior  (e.g.,  failure  to  follow  procedure).  A  higher  score 
indicates  poorer  performance.  Demerit  scores  ranged  from  0  to 
18 . 


Honors .  Recruits  who  perform  well  during  training  may  be 
assigned  a  position  of  responsibility  (e.g.,  lead  recruit). 

These  are  referred  to  as  honors  and  each  recruit  was  coded  as 
either  receiving  an  honor  or  not  receiving  an  honor. 

Inspections .  Various  levels  of  senior  personnel  conduct 
inspections  of  uniforms,  beds,  and  lockers  throughout  training. 
Recruits  are  expected  to  conform  to  standards  taught  during 
training,  and  each  inspection  (there  were  21  of  them)  results  in 
a  rating  of  satisfactory  or  unsatisfactory.  These  were  averaged 
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Table  5 

Correlations  Among  Training  Reaction  and  Performance  Variables* 


VARIABLE 

1 

2 

3 

4 

5 

6 

7 

8 

9 

1.  Demerits 

1 

1 

1 

t 

2.  Honors 

-.09 

— — 

3.  Inspections 

-.74* 

.07 

— 

4.  Academic  Tests 

-.28* 

-.01 

.24* 

— 

5.  Self-Rated 

Physical 

Performance 

.04 

.05 

-.03 

-.17* 

— - 

• 

6.  Self-Rated 

Overall 

Performance 

-.14* 

-.06 

.10* 

.32* 

.12* 

— 

7.  Overall 

Reactions 

.02 

.01 

-.02 

-.15* 

.30* 

.20* 

8.  Reaction  - 
Relevance 

.01 

-.02 

-.02 

-.17* 

.28* 

-.22* 

.93* 

9  Reaction  - 
Happiness 

-.02 

.03 

.01 

-.07 

.22* 

.07 

.75* 

.51* 

— 

“Listwise  deletion;  n=661. 
E<.01. 


to  yield  an  overall  inspection  score.  The  alpha  on  this  scale  is 
somewhat  low,  and  any  results  based  on  this  measure  should  be 
interpreted  with  caution. 

Tests .  Recruits  attended  lectures  and  were  tested 
throughout  training  on  information  regarding  shipboard 
procedures,  navy  protocol,  damage  control  procedures,  appropriate 
behaviors,  and  other  naval  procedures.  Four  tests  were 
administered  with  independent  content .  Scores  on  the  four  tests 
were  averaged  which  yielded  an  overall  academic  performance 
measure.  This  was  not  a  measure  of  learning  per  se,  as  no  base¬ 
line  information  was  available. 
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Self-Rated  Physical  Performance.  Throughout  training, 
recruits  take  physical  fitness  tests.  Unfortunately,  the  manner 
in  which  these  data  are  maintained  precluded  their  use  as  a 
measure  of  training  performance.  Instead,  trainees  were  asked  to 
rate  their  physical  performance.  This  single  item  indicated  how 
well  trainees  believed  they  performed  on  physical  tests  during 
training . 

Self-Rated  Performance.  Five  items  assessed  self-rated 
performance,  similar  to  those  that  assessed  performance  expecta¬ 
tions.  Trainees  were  asked  "how  well  did  you  perform  on...  [each 
major  performance  factor]."  The  five  items  addressed  physical 
performance  tests,  academic  tests,  uniform  inspections,  room 
inspections,  and  overall  performance.  These  items  were  based  on 
a  seven  point  Likert -type  scale  ranging  from  "Not  at  all  Well" 

(1)  to  "Extremely  Well"  (7)  with  "Moderately  Well"  (4)  as  the 
midpoint.  Factor  analysis  yielded  a  one-factor  solution.  The 
five  items  were  averaged  to  yield  an  overall  self-rating. 

Pr.Tnpr>g-i  Two  Composite  scores  were  developed.  The  first 

combined  demerits  and  inspections.  The  second,  referred  to  as 
"Overall  Performance,"  combined  demerits,  inspections,  honors, 
and  academic  performance.  To  create  these  scales,  all  scores 
were  converted  to  z- scores  (based  on  the  total  sample) .  Demerits 
were  multiplied  by  -1  so  higher  scores  would  be  better  and 
consistent  with  the  other  measures.  Scale  scores  are  averages 
across  the  two  and  four  measures,  respectively. 

Attrition.  As  an  indicator  of  the  Results  category  of 
training  effectiveness,  we  included  a  measure  of  attrition.  This 
was  defined  simply  as  whether  or  not  the  recruit  completed 
training. 

ANALYTIC  STRATEGY 

To  test  the  relationships  within  the  model,  and  to  allow  for 
an  assessment  of  the  relative  effects  of  several  independent 
variables,  a  series  of  hierarchical  multiple  regressions  was 
computed.  The  choice  of  variables  for  each  equation  was  based  on 
the  overall  model.  In  a  hierarchical  model  of  this  type, 
variables  may  serve  as  independent  variables  in  some  equations 
and  dependent  variables  in  others.  In  addition,  some  variables 
were  included  in  equations  that  did  not  reflect  links  in  the 
model .  This  was  done  to  assess  the  possibility  that 
non-hypothesized  relationships  might  exist. 

We  used  simultaneous  entry  within  steps  (a  more  conservative 
approach  than  stepwise  entry)  and  hierarchical  regression  between 
steps  whenever  a  theoretical  or  temporal  determination  of  order 
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was  possible  (cf.,  Cohen  &  Cohen,  1983  for  a  detailed  discussion 
of  this  analytic  strategy) . 

RESULTS 

Moving  from  left  to  right  in  the  model,  we  first  tested  for 
predictors  of  expectations/desires  and  pre-training  motivation. 
Next,  we  examined  factors  that  might  influence  training  reactions 
and  training  performance.  Finally,  we  examined  predictors  of 
post-training  self-efficacy,  post-training  motivation,  and  post¬ 
training  attitudes .  The  following  sections  delineate  the  results 
of  these  analyses. 

EXPECTATIONS  &  DESIRES 

According  to  the  model,  individual  non-ability 
characteristics  were  the  only  set  of  measures  (included  in  this 
study)  that  were  hypothesized  to  affect  expectations  and  desires. 
Therefore,  we  regressed  the  performance  and  training  expectation 
scales  on  all  the  individual  characteristics,  including  cognitive 
ability.  Table  6  presents  the  results  for  performance 
expectations  and  training  expectations.  The  regression  equations 
accounted  for  39%  and  24%  of  the  variance,  respectively. 

Academic  and  physical  self-efficacy  and  commitment  were 
positively  related  to  performance  expectations.  Physical  self- 
efficacy  and  commitment  were  positively  related  to  training 
expectations.  Surprisingly,  cognitive  ability  was  negatively 
related  to  training  expectations. 

Next,  we  regressed  training  desires  on  the  same  variables^. 
Table  7  presents  the  results  of  this  analysis.  The  adjusted  R 
for  the  equation  was  .28.  Physical  self-efficacy  and  commitment 
were  positively  related  to  training  desires. 

PRE-TRAINING  MOTIVATION 

Pre-training  motivation  was  regressed  upon  training  desires 
and  expectations,  and  all  the  individual  variables.  Table  8 
presents  the  results  of  this  analysis.  The  equation  accounts  for 
46%  of  the  variance.  Training  desires  and  training  expectations 
were  both  positively  related  to  pre-training  motivation,  as  were 
physical  self-efficacy  and  commitment. 

TRAINING  REACTIONS 

Expectation  fulfillment  was  hypothesized  to  predict  training 
reactions.  It  was  uncertain  whether  training  motivation  should 
be  related  to  training  reactions.  To  assess  the  relative  effects 
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of  these  two  variables,  they  were  simultaneously  entered  into  the 
equation  as  the  first  step  in  the  regressions.  Individual  varia¬ 
bles  were  hypothesized  to  be  only  indirectly  related  to  training 
reactions,  and  thus,  were  entered  simultaneously  as  the  second 
step  in  equation,  after  the  direct  effects  of  expectation 
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Table  6 


Regression  of  Pre-Training  and  Trainine  Expectations 
on  Pre-Training  Ability  and  Non-Ability  Variables 


Performance  Expectations  Training  Expectations 
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Table  7 


Regression  of  Training  Desires  on  Pre-Training  Ability  and 
Non-Ability  Variables 


Training  Desires 


Readjusted) 

F(df): 

E: 


.284 

33.08  (8,  640) 

<.0001 


Variables 

Beta 

t 

Family  History 

.02 

.49 

Academic  Self-Efficacy 

.02 

-  .47 

Sex 

.00 

-.07 

Intent  to  Remain 

.03 

.76 

Age 

.04 

1.17 

Physical  Self-Efficacy 

.21 

5.67” 

Cognitive  Ability 

-.07 

-1.65 

Commitment 

.40 

10.17” 
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Table  8 


Regression  of  Pre-Trainin£  Motivation  on  Training  Desires. 


Exnectations.  and  Individual  Variables 


274.74  (2,  646) 

<.0001 


Beta 

t 

.35 

8.99" 

.39 

10.13” 

Variables 


Training  Desires 


Training  Expectations  |  .39 


<.0001 


Variables 

Beta 

t 

Training  Desires 

.22 

5.61" 

Training  Expectations 

.34 

8.87“ 

Family  History 

.02 

.86 

Academic  Self- 

.04 

1.28 

Efficacy 

Age 

-.04 

-1.43 

Intent  to  Remain 

.00 

.29 

Sex 

-.06 

-1.73 

Physical  Self-Efficacy 

.15 

4.62“ 

Cognitive  Ability 

-.03 

-.79 

Commitment 

.21 

6.04" 
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fulfillment  and  motivation  were  removed.  Table  9  shows  the 
results  for  relevance/value  and  for  affect/happiness.  Both  steps 
were  significant  in  each  equation.  Over  30  percent  of  the 
relevance/value  variance  was  accounted  for  by  step  1,  and  10 
percent  of  the  affect/happiness  variance  was  accounted  by  step  1. 
Individual  variables  added  3%  for  relevance/value,  but  5%  for 
affect/happiness . 

Expectation  fulfillment  and  pre-training  motivation  were 
positively  related  to  both  training  reaction  measures.  Physical 
self-efficacy  was  positively  related  to  both,  and  intent  to 
remain  was  positively  related  to  affect/happiness  reactions.  Age 
was  negatively  related  to  both  reaction  measures. 

TRAINING  PERFORMANCE 

Ability  was  hypothesized  to  be  a  strong  predictor  of 
training  performance.  Expectation  fulfillment,  pre-training 
motivation,  and  all  individual  variables  were  simultaneously 
entered  into  the  regression  equation  for  each  performance 
measure.  Table  10  presents  the  results  for  each  of  the  four 
training  performance  measures. 

Forty-eight  percent  of  the  variance  was  accounted  for  in 
academic  performance,  with  the  vast  majority  of  it  attributable 
to  cognitive  ability.  Cognitive  ability  was  strongly,  positively 
related  to  academic  performance.  Older  trainees  and  women  also 
performed  better.  Academic  self-efficacy  was  positively  related, 
and  physical  self-efficacy  was  negatively  related  to  academic 
pej-f ormance .  Only  4%  of  the  variance  in  demerits  and  inspection 
performance  was  accounted  for;  cognitive  ability  was  positively 
related,  and  expectation  fulfillment  was  negatively  related. 

Because  the  physical  test  measures  were  unusable,  we  had  to 
rely  on  self-ratings  of  physical  performance.  Twenty  percent  of 
the  variance  was  accounted  for,  with  physical  self-efficacy 
accounting  for  the  largest  share.  Physical  self-efficacy, 
pre-training  motivation,  and  expectation  fulfillment  were  all 
positively  related  to  physical  performance.  Older,  and  female 
trainees  reported  lower  physical  performance .  Academic 
self-efficacy  was  negatively  related  to  physical  performance. 

Twenty- seven  percent  of  the  variance  was  accounted  for  in 
self-rated  overall  training  performance.  Academic  and  physical 
self-efficacy  were  both  strongly  positively  related  to  overall 
performance.  Expectation  fulfillment  and  commitment  were  also 
positively  related  to  overall  performance. 
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Table  9 

Re&ression  of  Training  Reactions  on  Expectation  Fulfillment 


Pre-TrainiiiE  Motivation,  and  Individual  Variables 


Training  Reactions 
Relevance/Value 


Training  Reactions 
Affect/Happiness 
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Table  10 


Rporpscinn  nf  Training  Performance  Indices  on  Expectation  Fulfillment 


Pre-Trainine  Motivation,  and  Individual  Variables 


Training  Performance- 
Academic 


Training  Performance- 
Demerits  &  Inspections 


Readjusted): 

F(df): 

E: 


.484 

61.28(10,  634) 
<.0001 


.043 
3.89  (10,  634) 
<.0001 


Variables 

Pre-Training 

Motivation 

Expectation 

Fulfillment 

Family  History 

Academic  Self- 
Efficacy 

Age _ 

Intent  to  Remain 
Sex 

Physical  Self- 
Efficacy 

Cognitive  Ability 
Commitment 


t 

Beta 

t 

1.54 

.01 

.25 

-.63 

-.10 

-2.47* 

-.69 

.03 

.73 

2.75" 

.05 

1.08 

3.96" 

.04 

1.08 

.92 

.03 

.73 

2.60” 

.00 

-.06 

-2.29" 

-.01 

-.27 

17.54" 

.19 

3.96" 

1.22 

-.01 

-.21 
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Table  10  (continued) 


Self-Rated  Performance  Self-Rated  Overall 

Physical  Tests  Training  Performance 
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POST -TRAINING  VARIABLES 

Expectation  fulfillment  was  hypothesized  to  influence  post¬ 
training  self-efficacy,  motivation,  and  attitudes.  Pre-training 
individual  characteristics  were  hypothesized  to  indirectly  influ¬ 
ence  the  same  post-training  variables. 

To  understand  the  impact  of  expectation  fulfillment  and  pre- 
training  individual  characteristics  on  post-training  self-effica¬ 
cy,  the  variance  attributable  to  pre-training  self-efficacy 
should  first  be  removed.  Similarly,  pre- training  motivation  and 
attitudes  should  first  be  removed  from  their  post -measures  prior 
to  further  analysis. 

It  is  possible  that  changes  in  self-efficacy,  motivation, 
and  attitudes  may  be  a  function  of  training  performance.  That 
is,  trainees  who  perform  better  may  report  that  they  "feel" 
better  as  well.  Therefore,  the  variance  attributable  to  training 
performance  should  also  be  removed  before  testing  for  the 
influence  of  expectation  fulfillment. 

To  test  these  effects,  we  conducted  a  four-stage 
hierarchical  regression.  In  stage  one,  the  pre-training  measure 
associated  with  the  dependent  variable  was  entered.  In  stage 
two,  the  overall  training  performance  measure  was  entered.  In 
stage  three,  expectation  fulfillment  was  entered.  Finally,  the 
remaining  pre-training  individual  characteristics  were  entered. 

Table  11  presents  the  results  from  the  hierarchical 
regression  for  physical  self-efficacy.  Pre-training  physical 
self-efficacy  accounted  for  47%  of  the  variance.  Training 
performance  did  not  account  for  any  additional  variance.  This  is 
not  surprising  since  the  training  performance  measure  of  physical 
performance  was  unusable  and  not  included  in  the  overall 
performance  measure . 

Expectation  fulfillment  was  positively  related  to  post 
training  self-efficacy,  although  it  accounted  for  very  little 
additional  variance.  Finally,  pre-training  motivation 
demonstrated  a  small,  positive  relationship. 

Table  12  presents  the  results  from  the  analysis  for 
post-training  academic  self-efficacy.  Pre-training  academic 
self-efficacy  accounted  for  21%  of  the  variance  in  post-training 
academic  self-efficacy.  Training  performance  was  positively 
related  to  post-training  academic  self-efficacy,  and  accounted 
for  an  additional  2%  of  the  variance.  Expectation  fulfillment 
was  non-significant  when  entered  alone,  but  demonstrated  a 
significant  positive  effect  when  entered  with  the  remaining 
individual  characteristics.  In  addition,  physical  self-efficacy. 
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Table  11 


Rearession  of  Post-Training  Physical  Self-Efficacy  on  Pre-Trainina 
Self-EfYicacv.  Training  Performance.  Expectation  Fulfillment,  Pre-Training  Motivation. 

and  Individual  Variables 


Step  #: 

1 

R': 

.472 

F: 

575.95 

E: 

<.0001 

Variables 

Beta 

t 

Pre-Training  Physical  Self-Efficacy 

.69 

24.00" 

Step  #:  2 

change:  .002 

F:  2.04 

d:  =.1534 


Variables 

Beta 

t 

Pre-Training  Physical  Self-Efficacy 

Training  Performance 

NS 

Step  #:  3 

change:  .008 

F:  9.39 

p:  <  .005 


Variables 

Beta 

t 

Pre-Training  Physical  Self-Efficacy 

.70 

24.25" 

Training  Performance 

-.03 

-1.11 

Expectation  Fulfillment 

.09 

3.06" 
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Table  11  (continued) 


Step  #: 

R-  change: 


Variables 

Beta 

t 

Pre-Training  Physical  Self-Efficacy 

.68 

20.53“ 

Training  Performance 

-.01 

-.43 

Expectation  Fulfillment 

.10 

3.39” 

Family  History' 

.03 

1.20 

Age 

-.06 

-1.94 

Intent  to  Remain 

.00 

.08 

Academic  Self-Efficacy 

-.06 

-1.88 

Sex 

.00 

-.17 

Pre-Training  Motivation 

.09 

2.70“ 

Cognitive  Ability 

.02 

.53 

Commitment 

.00 

.26 

Training  Performance 
Expectation  Fulfillment 


NS 
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pre-training  motivation,  and  cognitive  ability  demonstrated 
positive  relationships. 

Table  13  shows  the  regression  results  for  post-training 
motivation.  Pre- training  motivation  accounted  for  34%  of  the 
variance,  and  training  performance  explained  no  additional 
variance.  Expectation  fulfillment  accounted  for  an  additional 
10%  of  the  variance  after  removing  the  effect  of  both 
pre-training  motivation  and  training  performance.  Physical 
self-efficacy  demonstrated  a  small  positive  relationship  as  well. 

Table  14  reports  the  results  for  the  post -training 
commitment  regression.  Pre-training  commitment  accounted  for  20% 
of  the  variance,  and  training  performance  accounted  for  no 
significant  additional  variance.  Expectation  fulfillment 
accounted  for  an  additional  6%  of  the  variance.  Cognitive 
ability  also  showed  a  positive  relationship  with  post-training 
commitment . 

Table  15  presents  the  results  from  the  regression  of  post- 
training  intent  to  remain.  Pre- training  intent  to  remain  ac¬ 
counted  for  28%  of  the  variance,  and  training  performance  added 
no  significant  additional  variance.  Expectation  demonstrated  a 
small,  positive  relationship  with  post -training  intent  to  remain, 
as  did  physical  self-efficacy  and  pre-training  commitment.  Women 
demonstrated  lower  post -training  commitment  after  removing  the 
variance  for  the  previous  variables  in  the  regression. 

Attrition 


In  addition  to  the  regression  analyses,  a  discriminant 
function  analysis  was  computed  -to  determine  whether  it  was 
possible  to  predict  which  trainees  would  complete  training  based 
on  their  initial  responses  to  scale  items.  Statistically, 
discriminant  function  analysis  seeks  to  find  the  best  linear 
combination  of  predictor  scores  that  can  most  effectively  predict 
group  membership.  In  order  to  have  groups  of  relatively  equal 
size,  a  random  sample  of  150  subjects  was  drawn  from  the  original 
sample  (i.e.,  those  who  finished  the  training)  for  comparison  to 
175  in  the  sample  that  left  training. 

The  discriminant  function  analysis  revealed  that  four 
variables  appeared  to  be  significant  predictors  of  attrition: 
expectations  (p  <  .05) ,  self-efficacy  (p  <  .01) ,  commitment  to 
the  Navy  (p  <  .01),  and  pre-training  motivation  (p  <  .02) .  A  Chi 
square  coefficient  of  12.07  (df  =  4,  340,  p  <  .02)  indicated  that 
the  amount  of  variance  predicted  in  attrition  scores  (16%)  was 
statistically  significant. 
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Table  13 


Regression  of  Post-Training  Motivation  on  Pre-Training  Motivation.  Trainin 


Step  #: 

4 
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Table  13  (continued) 

change: 

.017 

Variables 

Beta 

t 

Pre-Training  Motivation 

.57 

15.46" 

Training  Performance 

.04 

1.08 

Expectation  Fulfillment 

.32 

10.32" 

Family  History 

.01 

.46 

Age 

-.04 

-1.21 

Intent  to  Remain 

.02 

.72 

Academic  Self-Efficacy 

-.04 

-1.21 

Sex 

-.06 

-1.78 

Physical  Self-Efficacy 

.11 

3.32" 

Cognitive  Ability 

-.03 

-.91 

Commitment 


.02 


.45 
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Table  14 


Regression  of  Post-Training  Commitment  on  Pre-Trainina 
rommitment.  Training  Performance.  Expectation  Fulfillment. 
Pre-Training  Motivation,  and  Individual  Variables 


Step  #: 

1 

R^: 

.200 

F: 

161.59  (1,  643) 

E: 

<.0001 

Variables 

Beta 

•  t 

Pre-Training  Commitment 

.45 

12.71** 

Step  #; 

2 

R-  change: 

.000 

F: 

.05 

E: 

=  .83 

Variables 

Beta 

t 

Pre-Training  Commitment 

Training  Performance 

NS 

3 

.064 

55.64 

<.0001 


Step  #: 

R-  change: 

F: 

E: 


Variables 

Beta 

t 

Pre-Training  Commitment 

.49 

14.21** 

Training  Performance 

.02 

.55 

Expectation  Fulfillment 

.26 

7.46** 

86 


Step  #: 
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Table  14  (continued) 

change; 

.031 

m 

Siq 


Variables 

Beta 

t 

Pre-Training  Commitment 

.37 

8.51*' 

Training  Performance 

.05 

1.28 

Expectation  Fulfillment 

.27 

7.67" 

Family  History 

-.05 

-1.42 

Age 

-.01 

-.40 

Sex 

-.05 

-1.33 

Academic  Self-Efficacy 

-.03 

-.74 

Physical  Self-Efficacy 

.03 

.80 

Intent  to  Remain 

.05 

1.43 

Pre-Training  Motivation 

.18 

4.29“ 

Cognitive  Ability 

.10 

.17 
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Table  15 


Step  #: 
R^: 

F: 

E: 


Step  #: 

change: 

F: 

E: 


Step 

R-  change: 

F: 

E: 


Re£ression  of  Post-Training  Intent  to  Remain  on  Pre-Trainina 
Intent  to  Remain.  Training  Performance.  Expectation  Fulfillment. 
Pre-Trainina  Motivation,  and  Individual  Variables 


1 

.282 

254.10(1,  643) 

<.0001 


Variables 

Beta 

t 

Pre-Training  Intent  to  Remain 

.53 

15.94“ 

2 

.001 

.57 

=  .4506 


Variables 

Beta 

t 

Pre-Training  Intent  to  Remain 

Training  Performance 

NS 

3 


.014 

13.15 

<.001 


Variables 

Beta 

t 

Pre-Training  Intent  to  Remain 

.54 

16.25“ 

Training  Performance 

-.01 

-.41 

Expectation  Fulfillment 

.12 

3.63“ 

88 
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DISCUSSION 

This  investigation  was  designed  as  an  initial  empirical  test 
of  some  of  the  key  constructs  and  relationships  in  the  model  of 
training  effectiveness  developed  here.  It  was  meant  to  assess 
the  usefulness  of  the  measures  and  the  model  for  understanding 
training  effectiveness.  In  particular,  the  data  collection 
focused  on  trainee  attitudes,  expectations,  and  motivational 
variables,  and  attempted  to  determine  if  they  demonstrate 
sufficient  utility  to  warrant  further  examination. 

In  general,  the  investigation  revealed  that  most  of  the 
measures  developed  and  used  in  this  effort  have  acceptable 
psychometric  qualities,  and  can  be  used  in  future  research. 
Additional  work  is  needed  on  some  variables. 

The  central  variables  of  trainee  attitudes ,  expectations , 
and  motivation  investigated  here  were  found  to  be  related  to 
other  variables  in  the  model,  and  appeared  useful  in  improving 
our  understanding  of  the  training  effectiveness  process.  In 
addition,  although  the  power  of  the  tests  may  have  yielded  some 
"trivial"  significant  results,  the  amount  of  variance  accounted 
for  in  most  of  the  regressions  was  quite  respectable,  even  in  the 
survey- to-archives  analyses.  The  encouraging  findings  of  this 
investigation  suggest  that  additional  research  is  warranted  to 
further  understand  the  role  of  these  variables  with  regard  to 
tiraining  effectiveness.  Below  we  discuss  key  findings  in  more 
detail . 

EXPECTATIONS,  DESIRES,  &  MOTIVATION 

Physical  self-efficacy  and  commitment  were  consistently 
related  to  expectations  and  desires.  Trainees  who  possessed 
higher  levels  of  physical  self-efficacy,  and  who  were  more 
committed,  had  greater  performance  expectations,  and  expected  and 
desired  more  from  the  training.  This  is  logical;  it  implies  that 
trainees  who  believe  that  they  can  perform  well,  and  those  that 
are  more  committed  to  the  organization,  want  more  from  the 
training.  Interestingly,  trainees  with  higher  cognitive  ability 
had  lower  training  expectations.  They  did  not  show  any 
differences  with  regard  to  desires.  Thus,  "smarter"  trainees 
hoped  for  the  same  things  in  training  but  had  lower  expectations 
than  other  trainees . 

Physical  self-efficacy,  commitment,  desires,  and 
expectations  were  all  positively  related  to  pre-training 
motivation,  with  expectations  demonstrating  the  largest  effect. 
Again,  this  makes  sense  since  those  trainees  who  believe  they  can 
do  well  (physical  self-efficacy) ,  are  committed  to  the 


90 


Technical  Report  93-011 

organization,  or  have  greater  desires  or  expectations,  are  also 
more  motivated  to  perform  well  in  training. 

TRAINING  REACTIONS 

Expectation  fulfillment  and  pre-training  motivation  were 
both  strongly,  positively  related  to  training  reactions.  Several 
individual  variables  also  demonstrated  small  effects. 

It  is  encouraging  that  expectation  fulfillment  was  so 
strongly  related  to  training  reactions.  When  the  training  meets 
or  exceeds  trainees'  expectations  and  desires,  they  view  the 
training  as  more  relevant,  and  feel  more  positive  about  the 
training.  This  is  support  for  using  the  "met  expectation" 
approach  to  studying  training  expectations,  as  suggested  by 
research  in  the  turnover  literature. 

TRAINING  PERFORMANCE 

As  predicted,  cognitive  ability  was  a  strong  predictor  of 
academic  performance  and  of  self-rated  overall  training 
performance.  Pre-training  motivation  was  positively  related  to 
both  self-rated  measures  of  performance. 

The  self-efficacy  measures  demonstrated  good 
discriminability .  The  academic  self-efficacy  measure  was 
positively  related,  and  physical  self-efficacy  was  negatively 
related  to  academic  performance,  and  vice-versa  for  physical 
performance.  Both  measures  were  positively  related  to 
self-rated,  overall  training  performance. 

Older  trainees  and  females  demonstrated  better  academic 
performance  than  younger  trainees  and  males.  Surprisingly, 
pre-training  motivation  was  negatively  related  to  academic 
performance.  As  academic  performance  was  not  a  true  measure  of 
learning,  we  would  have  expected  little  or  no  effect  for 
motivation . 

POST-TRAINING  SELF-EFFICACY,  ATTITUDES,  AND  MOTIVATION 

By  first  removing  the  pre- training  variance,  we  applied  a 
conservative  approach  to  examining  the  influences  on  the 
post- training  measures.  Regardless,  we  found  some  interesting 
predictors  of  the  post-training  measures.  For  example,  even 
after  removing  the  pre- training  variance,  expectation  fulfillment 
was  positively  related  to  all  five  post-training  measures.  While 
the  magnitude  of  the  effect  for  post-training  self-efficacy  was 
trivial,  its  effect  on  motivation  and  commitment  was  quite 
significant,  accounting  for  an  additional  ten  and  six  percent. 
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respectively  (after  removing  the  variance  from  pre-training  and 
training  performance  measures) . 

Other  small  effects  were  seen  for  pre-training  motivation, 
cognitive  ability,  physical  self-efficacy,  and  gender.  Training 
performance  was  significant  in  only  one  equation  (academic  self- 
efficacy)  and  the  effect  was  fairly  small.  Apparently, 
performing  well  in  training  is  not  the  key  factor  in  determining 
post-training  attitudes  and  motivation- -expectation  fulfillment 
is.  These  results  are  important  because  they  suggest  that 
motivation  after  training  (which  is  likely  to  affect  the  extent 
to  which  the  trainee  will  transfer  newly  acquired  skills)  is 
affected  by  the  trainees'  expectations,  and  the  extent  to  which 
training  met  those  expectations. 

ATTRITION 

While  the  results  from  this  investigation  regarding 
attrition  cannot  be  considered  definitive,  it  is  reasonable  to 
conclude  that  individual  and  organizational  factors  can  have  an 
impact  on  whether  a  trainee  completes  training.  In  the  present 
case,  expectations,  self-efficacy,  commitment,  and  motivation 
were  all  significant  factors.  These  results  clearly  require 
replication  and  extension.  If  supported,  they  suggest  that  pre¬ 
training  assessment  of  certain  factors  may  help  identify  trainees 
who  are  at  risk  of  not  completing  training.  Options  at  that 
point  may  include  remedial  programs,  or  depending  on  the 
situation,  denial  of  entry  into  training. 

SUMMARY  OF  DATA  COLLECTION  INTERPRETATION 

This  investigation  had  certain  strengths.  It  was  conducted 
in  a  field  setting,  and  incorporated  longitudinal  data  collection 
from  multiple  surveys  and  archival  data  sources.  Reliability 
estimates  were  available  for  almost  all  measures.  The  large 
sample  size  provided  ample  power  to  test  all  the  hypotheses. 
However,  the  effort  was  not  without  its  limitations. 

To  begin  with,  several  key  constructs  in  the  model  were  not 
testable  in  this  environment.  The  training  environment  was 
relatively  constant,  with  training  method  and  content  similar  for 
all  participants.  In  addition,  the  trainees  represented  new 
members  of  the  organization,  and  transfer  of  training  was  not 
assessed.  Thus,  no  training  or  organizational/situational 
variables  could  be  included  in  this  investigation. 

The  biggest  weakness  in  the  investigation  was  the  poor 
learning  and  training  performance  measures.  We  were  limited  to 
employing  training  performance  measures  currently  used  at  RTC. 
While  these  may  be  useful  for  administrative  purposes,  they  were 
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not  ideal  for  the  needs  of  the  current  research.  Since  no 
pre-learning  measures  were  collected,  there  could  be  no 
assessment  of  pre-post  learning  improvement.  In  addition,  some 
of  the  performance  measures  (i.e.,  honor,  demerits)  were  used  for 
motivational  purposes  as  much  as  for  evaluation,  and  the  physical 
performance  measures  were  unusable.  Despite  the  potential 
problems  with  the  training  performance  measures,  this 
investigation  still  revealed  some  interesting  findings.  However, 
any  follow-up  research  should  consider  the  quality  and 
availability  of  learning  and  performance  measures  that  are  avail¬ 
able  . 


On  the  basis  of  this  effort,  we  can  strongly  encourage 
additional  research  in  this  area.  There  is  some  additional  work 
to  be  done  in  scale  development,  particularly  in  the  areas  of 
motivation  and  expectations.  Furthermore,  future  research  should 
examine  measures  of  trainee  attitudes,  motivation,  training 
expectations,  and  self-efficacy.  The  use  of  a  training 
fulfillment  measure  (as  developed  in  this  investigation)  holds 
particular  promise  for  improving  our  understanding  of  the 
training  effectiveness  process.  Although  this  investigation 
merely  scratched  the  surface,  we  found  some  evidence  for 
processes  related  to  training  effectiveness.  Further  research  is 
warranted . 
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CONCLUSIONS 

This  project  met  all  of  the  goals  noted  at  the  beginning  of 
the  report .  We  developed  a  conceptual  model  of  training 
effectiveness  that  helped  to  integrate  research  results  from 
diverse  literatures.  On  the  basis  of  that,  we  conducted  an 
empirical  field  investigation  to  assess  the  usefulness  of  the 
model.  In  the  empirical  investigation,  we  identified  and 
developed  some  useful  measures,  and  lent  some  support  to  the 
proposed  model.  However,  as  noted  earlier,  it  is  impossible  to 
test  all  variables,  relationships,  and  hypotheses  in  one  study. 
Some  measurement  issues  and  research  needs,  as  revealed  by  the 
literature  review  and  the  empirical  study,  are  noted  below. 

MEASUREMENT  ISSUES 

1)  Most  of  the  previous  research  has  examined  training  reactions 
as  a  unidimensional  construct.  We  believe  that  it  is  multi¬ 
dimensional  and  found  some  preliminary  support  for  that  belief. 
Additional  work  is  needed  to  develop  sound  scales  for  assessing 
training  reactions.  Item  development,  factor  analysis,  and 
validation  work  are  all  necessary. 

2)  Training  expectations  have  shown  some  promise  in  clarifying 
the  training  effectiveness  process.  However,  the  factor  analytic 
work  in  our  study  suggested  that  there  are  different  sub-groups 
of  training  expectations  (e.g.,  pertaining  to  training  method, 
pertaining  to  challenge) .  One  of  the  subscales  demonstrated  poor 
reliability.  Future  work  is  n(ieded  to  improve  this  measure  to 
allow  for  an  examination  of  subscales.  In  particular,  does 
expectation  fulfillment  for  one  subscale  have  greater  impact  than 
for  others?  Scale  development  work  is  needed  first. 

3)  We  believe  that  self-efficacy  is  a  key  concept  for  understand¬ 
ing  training  effectiveness.  The  academic  self-efficacy  scale  did 
not  hold  together.  Further  work  is  needed  on  that  measure.  In 
addition,  the  specificity  of  the  self-efficacy  measures  should  be 
examined  as  well. 

4)  We  treated  training  motivation  as  an  overall  concept,  measura¬ 
ble  over  time.  Future  research  should  attempt  to  develop 
sub-scales  of  training  motivation  specif ically , targeting  motiva¬ 
tion  to  attend,  motivation  to  learn,  and  motivation  to  transfer. 
These  measures  might  demonstrate  better  predictability  at  appro¬ 
priate  points  in  the  model . 

RESEARCH  QUESTIONS /NEEDS 

1)  What  determines  the  transferability  of  self-efficacy  across 
tasks?  What  are  the  antecedents  and  consequences  of 
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self-efficacy  as  it  relates  to  training?  How  does  improved 
self-efficacy  relate  to  transfer?  We  would  hypothesize,  that  in 
transfer  environments  with  poor  situational  favorability  (e.g. , 
poor  supervisor  support) ,  high  self-efficacy  is  a  more  critical 
predictor  of  transfer.  This  is  based  on  Bandura's  (1986) 
clinical  work  with  self-efficacy. 

2)  What  are  the  antecedents  of  training  expectations  and  desires? 
To  what  extent  are  they  formed  through  organizational  experi¬ 
ences?  Which  organizational  experiences  are  most  salient?  How 
do  supervisors  and  co-workers  affect  training  expectations? 

3)  Does  the  purpose  of  training  (e.g.,  reward  versus  punishment; 
to  improve  in  current  job  versus  prepare  for  promotion)  affect 
training  motivation  and  desires? 

4)  There  has  been  a  little  research  on  the  organizational  and 
situational  factors  that  facilitate  or  inhibit  transfer.  Further 
work  is  needed.  Perhaps  this  research  should  begin  with  rich, 
qualitative  data  collection  based  on  diary  keeping  and  content 
analysis,  supplemented  by  interviews.  What  exactly  do  peers  and 
superiors  do  that  is  related  to  change  on  the  part  of  trainees? 

5)  What  is  the  impact  of  conditional  knowledge  on  transfer  of 
training?  Most  of  the  studies  that  examined  conditional  knowl¬ 
edge  did  so  in  an  academic  or  clinical  environment.  Do  similar 
results  occur  in  organizational  settings?  Does  it  work  because 
of  its  effect  on  motivation  to  transfer? 

6)  What  is  the  relationship  among  the  six  categories  of  training 
outcomes  in  our  proposed  typology?  Under  what  circumstances  is 
the  typology  hierarchical? 

7)  What  are  the  relative  effects  of  ability  and  motivation  on 
training  effectiveness?  This  should  be  tested  in  research  using 
pre-  and  post -measures  of  training  performance  and  learning.  We 
would  hypothesize  that  ability  is  related  to  both  pre-  and  post¬ 
measures,  but  that  motivation  is  related  to  changes  from  pre  to 
post.  In  addition,  are  there  interaction  effects,  and  if  so,  are 
these  contingent  upon  characteristics  of  the  training  task? 

8)  Further  work  is  needed  that  examines  aptitude  training  inter¬ 
actions.  When  do  they  occur? 

9)  Do  different  forms  of  training  needs  analysis  yield  better 
result s/organizational  effectiveness? 

10)  How  does  the  training  effectiveness  model  apply  to  team 
training?  Cohesiveness  is  a  key  to  team  performance  (Salas, 
Dickinson,  Tannenbaum,  &  Converse,  1992) .  Do  trainee  attitudes 
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play  a  more  important  part  in  this  context?  Are  trainees'  expec¬ 
tations  and  motivations  shaped  by  teammates  reactions  during 
training?  If  teammates  express  boredom,  lack  of  interest,  or 
argue,  does  this  undermine  training  effectiveness?  Do  teammates 
provide  feedback  that  can  motivate  other  teammates  to  learn  and 
transfer?  How  can  this  be  encouraged?  Some  team  training  is 
designed  to  enhance  "teamness."  How  do  we  measure  this  construct 
before,  and  after  training,  to  determine  attitude  change?  In 
measuring  reactions  to  team  training,  we  must  consider 
variability  as  well  as  averages.  What  do  outliers  tell  us  about 
the  team  training  process? 
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RECOMMENDATIONS 


IMPLICATIONS  FOR  TRAINING 

One  of  the  objectives  of  the  present  research  was  to  provide 
a  basis  for  generating  principles  of  training  system  design  that 
will  maximize  the  chances  that  training  will  be  successful.  In 
particular,  we  sought  to  begin  providing  guidance  for  training 
system  designers  through  an  understanding  of  those  factors  that 
affect  training  effectiveness.  From  the  current  findings,  the 
following  initial  principles  can  be  offered  in  this  regard: 

1)  The  level  of  self-efficacy  of  trainees  should  be  assessed 
prior  to  training. 

2)  Remedial  training  to  raise  self-efficacy  levels  prior  to 
training  will  enhance  probability  of  positive  training  outcomes. 

3)  Trainees  should  be  led  to  have  realistic  expectations  for 
training.  Interventions  to  meet  this  objective  should  be 
designed . 

4)  Interventions  designed  to  increase  trainee  commitment  to  the 
organization  will  enhance  the  likelihood  of  successful  training. 

5)  Efforts  to  improve  trainee  motivation  prior  to  training  can 
lead  to  better  training  outcomes. 

Overall,  these  results  imply  that  no  matter  how  wel] 
designed  a  training  system  is,  training  effectiveness  will  not  be 
optimized  without  a  consideration  of  pertinent  individual  and 
organizational  factors. 

SUMMARY 

In  summary,  a  process  view  of  training  effectiveness  should 
yield  dividends  in  terms  of  improved  understanding  of  crucial 
training  variables,  and  improved  training  outcomes.  The 
development  of  diagnostic  measures  based  on  the  key  components  in 
the  model  can  serve  as  training  evaluation  tools  to  specifically 
target  where  interventions  in  the  training  process  are  needed. 
Hopefully,  the  framework  we  have  developed  can  guide  future 
research  and  continue  to  increase  our  understanding  of  why 
training  is  effective. 
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COORDINATION 

This  effort  has  been  coordinated  with  a  number  of  agencies 
and  organizations.  In  particular,  the  work  was  briefed  in  detail 
at  Training  Technology  Technical  Group  (T2TG)  meetings  to 
representatives  from  the  Naval  Personnel  Research  and  Development 
Center  (NPRDC) ,  the  Air  Force's  Armstrong  Lab,  and  the  Army 
Research  Institute  (ARI) . 

In  addition,  a  paper  presentation  representing  this  effort 
was  presented  at  the  14th  annual  Interservice/Industry  Training, 
Simulation,  and  Education  Conference  (I/ITSEC)  in  San  Antonio, 
Texas,  in  November,  1992.  The  work  has  also  been  coordinated 
with  researchers  at  several  universities,  including  the  State 
University  of  New  York,  Pennsylvania  State  University,  and 
Michigan  State  University. 
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